That Counterintuitive Moment

I explored the relation between data and intuition in my previous blog. Here I’ve pondered the concept of coutnerintuition and the importance of perseverance in a pursuit. I think a discovery in science is that unique moment when one’s intuition intersects with a phenomenon, which subsequently gets rationalized either mathematically or through an experiment. Some of those discoveries happen to get counterintuitive too i.e. they tend to defy (sometimes completely reversing) a previously held intuition, only to evolve into a new intuition providing us with that extra bit of knowledge. In the subsequent paragraphs, I recall a few instances that I think were/are counterintuitive.

Most of us are aware of the flat world theory that prevailed prior to 5th century BC. When proposed, the concept of a spherical world was difficult to grasp for many. Even the scholars during that time were unable to accept it until they began see empirical evidence in its favor. I think moving to the spherical world would have been a counterintuitive moment for those who had embraced the flat world theory completely.

The second one is Heisenberg’s uncertainty principle. This principle asserts the fundamental limit in precision with which certain pairs of physical properties (position and momentum) of a particle can be measured. This can be demonstrated through an experiment wherein; a laser beam is made to pass through a slit (of a variable length) and projected on a screen. The image shrinks as we narrow the slit, which is intuitive, but beyond a specific point (when the slit is narrowed to less than a 100th of a cm), the behavior reverses – the image actually begins to widen as we narrow the slit further. I think for someone who has been exposed to classical physics alone, this behavior could sound counterintuitive.

My last example is related to the game of cricket. For a long time the fast bowlers of the game were used to the intuition of the conventional swing. The cricket ball when bowled fast swings in the direction of the side that is relatively rougher. For decades fast bowlers exploited this by polishing only one side of the ball in the course of the game to maximize the possibility of a swing, creating trouble to the batsman. The intuition was completely justified and aerodynamically rationalized, until the concept of reverse swing was observed a few decades back. Reverse swing is observed when the cricket ball is bowled even faster (say 80 mph). At this critical speed, the intuition is defied and the ball also swings in the direction of the polished side. Understandably this counterintuitive behavior took some time to settle in the cricketing world.

In all above cases I sense someone’s (the first observer of each of the above phenomenon) relentless pursuit to evolve an intuition. I happened to stumble upon this interesting perspective by Richard Feynman on what Physics is all about. Feynman compares Physics to a grand game of chess played by God and humans trying to interpret the rules of it by just watching a few moves here and there. All of a sudden, pursuing to intuit appears more important than the act of intuiting itself.


Data & Intuition

Few years back, my friend explained how he understood the word ‘intuition’. Per him, the word is a possible derivation from ‘in-tutor’, which refers to the ‘tutor’ or the ‘guide’ inside each one of us. If that were to be true, we reflected why we find it difficult to follow our inner tutor while making important decisions. I think the reason we get jittery about following our own intuition is intuition is un-rational. What Mahatma Gandhi calls as the ‘inner voice’ in his autobiography is intuition (I believe).  Albert Einstein calls an intuitive mind a sacred gift that needs to be honored. Per Steve Jobs, intuition is the most important thing, even more important than intellect.

While intuition is instinctive, the other way to decide is through reason that happens primarily by analyzing data or the information we have at hand. Many business leaders have stressed the importance of data driven decisions as well. A quote of N.R. Narayana Murthy goes like this – In god we trust and everyone else come with data. Recently read an article about Marissa Mayer’s decision to ban Yahoo employees from working from home. The article cites Mayer’s obsession with data and metrics to help her decide.

While both forms of decision making (intuitive and data driven) look absolutely pertinent in our daily lives, this blog is an attempt to resolve a personal dilemma i.e. what should prevail when one is forced to choose? As a BI practitioner, I can fully appreciate the importance of data and metrics. I continue to see it add value in business and in my personal life. But I’m sure all of us would have crossed those unique moments in life through gut, hunch or intuition. I also believe intuition is more fundamental an experience to inference.

I recently stumbled upon this email, posted online by Andrew Mason. Many of us are aware of this charming goodbye email written by the founder of Groupon. In his letter, he shares an important wisdom. His regret in his own words – My biggest regrets are the moments that I let a lack of data override my intuition on what’s best for our customers. To me, his statement captured the importance of both intuition and data. As I think of it, every data driven scientific decision that we experience is someone’s intuition in the form of a mathematical model, algorithm etc. I think one should strive to decide by intuition. Technology solutions built around data and metrics can help serve 2 purposes here – (1) either to execute those decisions or (2) to help the decision maker stay clear of any doubt he/she might have during the process. I’ll sign off with another quote of Mahatma Gandhi – You must try to listen to the inner voice, but if you will not have the expression ‘inner voice’, you may use the expression ‘dictates of reason’, which you should obey.

Overriding thoughts & the inevitable simplicity

Years ago were a time I was worried about the future and for a brief period had lost my ability to stay happy. Looking at my state, my friend advised that every thought is our own creation and we have absolute control over them implying happiness is my own choice. Recently, another friend of mine  shared his intuition that material existence involves navigating ones collection of nested thoughts. One thing appears clear – thoughts are inevitable as we get through this life.

While we can think at will i.e. while we can choose our next thought, our choice is bound by what we remember. We wont know what we forget at the moment right ? And am sure all of us do forget something all the time. Also once in a while, all of us would have experienced something ‘out of the blue’ or through instincts i.e. those ‘out of context’ thoughts. This makes me think and even conclude that the arrival of the next thought is elusive over which we will never have absolute control over. Now, that gets me nervous – the lack of control over our own thoughts which eventually drive our actions. But an appropriate subsequent thought is going to hold importance especially during moments of uncertainty. So how do we handle this paradox?

While we cannot control, I think we can influence. Influencing happens through proactively conditioning the mind with the thoughts that we would like to experience. Through such conditioning, we improve the probability of us experiencing the right thought at the appropriate time. Now, if we decide to condition, what could be our selection of those thoughts? Thoughts that could come handy and help us decide during uncertain moments could be one good choice. Especially those thoughts that are driven by ‘will’ which override the convenient ones for the not so convenient ones for a deeper purpose. I’ll call these crucial thoughts as ‘overriding’ thoughts. As one embarks on such a conditioning pursuit, he/she will most likely realize a constraint that such overriding thoughts can only be those that are easy to remember. To remember, the thought has to be simple and a fundamental one that finds frequent applicability. I have an  intuition that such thoughts can get nothing other than fundamental qualities (both positive and negative) around decency, discipline, spontaneity etc. It does appear that simplicity (in both thoughts & actions) is inevitable in any pursuit, only that the pursuit has to be genuine.