Life Of Navin

Random Musings, Random Bullshit.


Solivagant Fernweh

... and I'm off! :D

Network effects and social media

Wikipedia defines Network effect as:
[...] the effect that one user of a good or service has on the value of that product to other people. When a network effect is present, the value of a product or service is dependent on the number of others using it.
What this basically means is that a network becomes more valuable to each individual user when the overall number of users of the network increases.

Unfortunately, for social networks, there tends to be a disturbing trend of having signal to noise ratio diminish almost exponentially when the number of people using it increases. The fact that most networks these days follow the "acquire them first, charge them later" strategy of expansion, makes it almost impossible to stick with a social network past it's nascent growth stage.

I personally have seen this effect in action quite a few times over the last few years.

Twitter: I was one of the early Indian community users on twitter and have fond memories of the close-knit nature of the network. There used to be a time when Twitter was actually seen as a voice of change. Today, twitter is reduced to an unedited stream of constant, random banter. The fact that news channels make use of snippets from Twitter has led to an army of loud noises, most of whom use twitter to blow steam rather than have a discussion, which is what I always saw Twitter as. The amount of vitriol on Twitter is toxic. The irony is that the quality of stuff has degraded on Twitter even though the number of people I follow has not really risen by much. Voices of reason are now replaced with comments of fanaticism, life experiences are replaced with silly oneliners. Sigh.

Facebook: I never really have been a fan of Facebook, even having quit the site for some time, and never really having completely gotten back to it since then. I'm much more of a commenter on the site than a sharer (My last status update, if they're still called that, dates back to mid 2012), except for some college groups. Facebook has never really helped me "stay in touch" or "get in contact" with people. Maybe I'm using it wrong, but I honestly prefer just using mail or phone or in-person meetings. Facebook has always had a content problem, and it's personalisation seems targeted to depress people. As the quote goes:
The problem with Facebook is we compare everyone’s highlight reel to our behind-the-scenes
My mentor at Dreamworks had warned that Facebook goes through 2 major events. All your friends getting married (followed by their honeymoons), followed 2-3 years later by baby pics. I laughed at this back then, but my current Facebook feed is eerily flooded by the former.

HackerNews: OK, I admit, I'm a HN addict. I just can't get enough of the site. I use hckrnews as a proxy to ensure I get only the best content on the site, and I'm pretty blown away by the quality of stuff that makes it to the homepage. But if I ever visit the actual discussion on the actual HN site, there's so much elitism on there. The problem I see with HN is a trend that's disturbing amongst developers and the tech community at large: Pointing out fault in everything. Doesn't matter if it's a hobby project or something that someone spent the last 3 years of his/her life on. The moment it hits HN, there will be a upsurge of "That's stupid" or "That would never work" type of comments, usually addressing a completely different context than the original post. Just see the comments Dropbox got on it's launch. And then we wonder why the tech community doesn't have a more diverse community. Baah. I've been blessed to have some amazing people mentor me directly and indirectly over the years I've played with computer systems, and I can promise that this attitude problem with the tech community at large would lead to disaster if not fixed ASAP.
Everyone on HN... All the time
Quora: The biggest offender of the lot. Oh Quora, my Quora, why did you abandon me so? Quora's biggest USP was the quality of content on the site. And then one day, Quora decided to make questions free to ask. Boom. Headshot to Quora post quality. The day I knew Quora was, for all practical purposes, dead, was the day I was A2A "If Pakistan kidnaps Sachin Tendulkar, what will India Government's response be?". Personally for me, Quora has 2 problems: Firstly, Quora content has taken a turn from the factual to the experiential: "How did China become a dominant manufacturing hub?" is replaced with "How does it feel knowing your phone was made by child labourers in China?" and so on. Experiential content is very subjective. For eg. I love some of the answers given by the San Quentin inmates, as it helps me understand the mindset of people whom society considers dangerous, but honestly most content is a blatant mockery of the goal of the site. A2As of interview experiences, being smart, GATE preparations et al, are commonplace and extremely irritating. Which brings me to the second point: The Indian problem. Quora has steadily become a place where Indians (specifially, Indian Engineering/MBA students) ask and discuss Indian problems. Questions about JEE, CAT, Career Advice, Personal experiences, company comparisons etc have practically reduced Quora to a exam prep site. I blogged about this before, and unfortunately the problem has just gotten worse. And don't even get me started on the meme-ification of the site.

All in all, no social network seems to have any silver bullet solutions to this problem of the masses. This problem, for some reason, is very reminiscent of the tragedy of the commons. While HN lacks diversity due to overt  moderation, Twitter and Quora seem to suffer from too much banality due to the cost/punishment for banter being negligible. I don't know if it's too much to ask for, but my hunt for an online community which is self-moderating, intellectually interesting and diverse across multiple axes continues. 

Review: Pycon India 2015

Another year, another Pycon India. Pycon India 2015 was held in Bangalore at the (usual) NIMHANS convention center. Keeping in tradition with the last few years, I was part of the volunteer team for the event (Pycon India is a complete community effort). The event comes at the fag end of pretty messy period for the Indian Python community and the last few months have been dotted with tonnes of arguments, unconstructive criticism of community members, widely varying views on how grassroots level python events should be organised (in schools, colleges and so on), and an extremely rigid, unnecessarily bureaucratic approach to things by the PSSI. And somewhere, I think this all reflected on the quality of the conference itself. This was the fifth Pycon India I've attended, and I must say, this was the first year when I didn't have an absolute blast. 

Multiple things were off the mark at this year's Pycon. The talk selection, which has always been a topic of contention, basically took a nosedive this year with most of the talks being either surface level at best, and downright boring at worst. This was perhaps the first time I excercised the law of two feet, and chose to walk out a couple of talks which I felt were a waste of time.

The internet and video recording, which was usually handled by the guys at HasGeek, was not this year, and the difference in quality in both of these was immediately evident. HasGeek has always set the bar high with it's events, and their absence at Pycon was clearly felt.

I usually do a list of my 5 favorite talks at conferences, but the speakers were so dissappointing, I could come up with only 3:

1) Solving Logical Puzzles with Natural Language Processing by Ashutosh Trivedi: Ashu, who was my classmate at IIIT-B, delivered a crisp talk on the state of the art of NLP, and how machine learning techniques can be leveraged to bring together more meaning and logical consistency to plaintext sentences. While NLP was a popular topic amongst speakers at the event, Ashu's talk was by far the most researched and well presented.

2) How to detect Phishing URLs using PySpark Decision Trees by Hitesh Dharmdasani:
Hitesh delivered a fun talk on the different characteristics of phishing URLs and the work that went into creating a PySpark decision tree to detect if a url is valid or a possible phishing URL. Work like this goes a long way in ensuring that people don't get fooled in this day and age where even tech savvy people are caught unawares by social engineering and phishing attacks.

3) Keynote: Education Education Education by Nicholas H.Tollervey: Nicholas, who was one of the keynote speakers at Pycon India 2015, gave a really thoughtful talk about education and python. He spoke about teaching as a means of self improvement as well as community involvement, and also did a cool demo of the micro:bit, which is an ARM based RasPi like system which is being used for computer education for kids in the UK.

The Python logo made from Rubik's Cubes at Pycon

I couldn't exacly put my finger on it, but something definitely felt missing at this Pycon. All in all, there was a sense of too much familiarity at Pycon this year. So I'm glad that next year Pycon is happening at a brand new location, New Delhi. The pythonistas from New Delhi have played a firm role in organizing Pycon over the last few years, and I'm sure that they'll cook up something outstanding next year. Definitely looking forward to it :)

And that has made all the difference ....

Today morning, I came across an image which struck me on a deep personal level. Here is the image:

The human mind works in a funny way. Do something everyone else does for long enough, and you'll soon enough start feeling the need to stand out in some way from the rest of the pack. On the other hand, stand out in the limelight for too long and you soon feel the need to go running back to the safe comfort of the very same rat pack you despised not too long ago.

For a long time, I've always tried to do things in a way that would make sense to everyone else at some point in the future rather than at the moment I make them. Some of these gambles have worked beautifully, others not so much. But as they say, every failure is a stepping stone to success. Life has been a beautiful journey so far, with my friends and family having been super supportive of the choices I've made, even when they do not fully agree with them.

But I need to come out with a confession: I'm pretty lost right now. The future is now a term I use to refer to any day which is not today, with no certainty of decisions I've made over the past week, month or year for that matter. Don't get me wrong, lots of awesome things have happened over the last year (people who know me well would attest to that :P). Awards have been won, hackathons have been hacked through, lots of amazing people have been met, tonnes of awe-friggin-tastic places have been visited, and laurels have been achieved both in the academic world and outside of it. But the question still remains, "Is this all worth it?". Are friends who chose more conventional choices when it came to jobs (Infy/Cogni/Wipro), or lifestyles (Choosing to live with family/closer to family), or education (MBA post-Engineering, followed by a big salary consultant job*) really that much worse off than me and others like me who preach the path less taken? And the honest answer is: I don't know.... yet.

Steve Jobs famously said "You can't connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future." But trust me when I say this: This is much easier said than done. The dots sure take their own sweet time to connect. And until they do, you really have no way of knowing how right or wrong your path has been.

In the end, maximising personal happiness remains a guiding star that resolves all those difficult to make decisions. Asking myself the question "If I do X, will I end up being happier in life?" before making big decisions has usually helped. Yes, I've passed up on things that a couple of years ago, would have made me jump with joy but now don't seem to be worth it simply because of the answer to this question ( LOL... yes, dear person who knows me, I am talking about the things you think I'm talking about :P ).

The bigger picture is still a big haze, but I'm happy that every once in a while, the universe gives hints that everything will be alright. And I just want to let anyone else who's in the same boat right now know that the seemingly lonely path we've chosen is not so lonely after all. Being lost by choice is a lifehack in itself, and while I definitely want to get to a point where everything suddenly makes sense**, I'm in no hurry to get there as long as my gut tells me I'm in the right direction.

* I just watched House of Lies, and I need to know: How much of that lifestyle is actually based in reality? 
** Does this even ever happen?

Outside the box.

We live in a society exquisitely dependent on science and technology, in which hardly anyone knows anything about science and technology. - Carl Sagan

Protip: Check this page tomorrow :)

Review: FifthElephant 2015

I attended the 2015 iteration of FifthElephant as part of the Bloomreach contingent. This was special because this was the first time I attended a conference as a sponsor rather than just as a regular participant/volunteer. The whole event was super fun, meeting up with lots of interesting people (both at the booth and otherwise), spending time digging deep into technology stacks of different companies and grokking tonnes of stuff which was all arranged in my head under the *stuff I read on hackernews but need to explore more* category. While I did not get to attend as many of the talks as I would have, I still managed to catch a tonne of interesting talks as well.

My favorite talks from FifthElephant 2015 include:
1) Vedang Manerikar's Dead Simple Scalability Patterns: Vedang is someone I know and admire since quite a while, be it as a developer or as a fellow Mahabharata geek, and staying true to expectation, he delivered a brief, to-the-point talk on scalability patterns. Scaling a system isn't sexy or cool but takes a well crafted approach. This was probably the first talk I attended at the conference, and it was a good refresher on stuff that should be done, but usually is ignored until something goes wrong. [video]

2) Himadri Sarkar's Approximate algorithms for summarizing streaming data: Himadri's talk was a crisp talk filled with nuggets of wisdom on how to do stuff which seems simple on much smaller datasets on datasets that are extremely huge, where you can trade exact correctness for constant memory and inexpensive operations. Starting with Bloom Filters, and heading to more advanced data structures, he did a really good job of explaining the logic behind these structures. I've been following developments like HyperLogLog since Antirez blogged about it, and having read about approximation algorithms through the lovely book by Williamson and Shmoys in college, this talk resonated with me quite a bit. [video]

3) Dr. Shailesh Kumar's keynote: Towards Thinking Machines: Whattay keynote! Dr. Kumar basically taunted everyone at FifthEl into asking the question "Why don't we have thinking systems yet?". It questioned basic premises of thinking such as the meaning of creativity, the concept of thinking, and the roots of semantics. When a system interprets that a restaurant is cozy based on restaurant reviews, is it simply doing NLP and using language rules, or can we reach a stage where machines can actually understand what cozy means? Is deep learning, and throwing a million images of cats at a system to get it to understand what features make a cat the right way to approach learning? Is this how humans think (spoiler: No). The work I did in WSL was very much aligned with this, so I completely loved this talk. The moment he said triples, I had the widest grin in the room :P [video]

4) Viral B. Shah's The many ways of parallel computing with Julia: Julia is a high level language that has been taking the world by storm for being super useful for high performance computing, and who better to speak about it than Viral, who is one of the co-creators of the language. Right at the start he mentioned that this talk was also going to be a State of Julia talk, and he did a great job of getting people interested. Julia is one of those languages that has a very strong research background and as such is generating massive interest. Definitely need to look more into Julia. To get started, you can use the online Julia console JuliaBox [video]:

5) Amit Kapoor's Visualising Multi Dimensional Data [video]: Without doubt the most beautiful slide deck at FifthElephant 2015. Amit used a lovely slide deck to speak about data visualization in 4 quadrants: Small, Big, Large and Wide. He went through challenges and visualization techniques for each of these along with the intricacies that different visualization techniques bring in. I learnt about some visualization techniques that I had never heard about before, such as Trellis plots and Star plots. The importance of interaction in visualizations was also stressed upon. Hat tip for starting off with a mention of Flatland

6) Vinodh Kumar's Building a E-commerce search engine: Challenges, insights and approaches:  Vinodh did a lovely talk on the different challenges in designing and scaling a Ecommerce Search engine. He started off with a beautiful analogy which compared searching the world wide web a la Google vs. searching a Ecommerce site a la Amazon was like comparing searching the landmass of Africa vs. searching the area of Cubbon Park, and then went on to explain why Ecommerce search brings in it's own challenges, and semantics and the problem of choosing "Best among equals". Lots of context, lots of importance on ranking, lots of person-to-person differences, and how exactly you'd come up with a fair algorithm to show results. He ended up with the quote "To search Cubbon Park, you need to learn from techniques used to search Africa". Touche! [video] Disclaimer: Vinodh is CTO of Bloomreach, where I work

A talk which was not exactly technical, but still a fun talk to attend was Steven Deobald's Two Years Wiser: The Nilenso Experiment. Steven spoke about Nilenso, a worker's cooperative, which means that everyone working at the company owns the company. He began by describing the talk as "Disaster Porn", and went deep into how the company learnt, through others' experience and through their own mistakes with a company structure which is very unorthodox, to put it lightly. I loved the fact that Nilenso, as a company, ideologically supports openness and sharing and aren't afraid to not just make mistakes, but take steps to ensure no one else makes the same mistakes as them. [video]

Part of the Bloomreach contingent at #fifthel

On the whole, FifthElephant was an excellent event to be at. Tonnes of brain food, lots of interesting discussions, and overall a lot of fun. Kudos to HasGeek, who have matured to the stage where events like this usually go off without any hitch. The guys from SumoLogic were super fun to be around throughout the conference. Same applies to the guys from Aerospike as well. Anand from Mad Street Den also popped by for some fun discussions. The random discussions that happened outside of the conference halls were just as awesome as the talks going on inside. Definitely a event I look forward to being a part of again next year. :)



Finally after all these years, here's to the beginning of what was there, what is there and hopefully what will remain!! So here are my thoughts & words -Online!!