AI, Stress and Information Juggernauts

I was looking at all the holiday cards I received from family and friends this year thinking “man, I love staying in touch with people but don’t have a cute kid. Hell, I don’t even have an ugly one.” One might not normally think of fun discoveries in neuroscience and artificial intelligence as “cute,” but they’re what fire me up most so I figured it’s as good an excuse as any to keep in touch.

To that end, the geeky shit I’ve been loving since my last newsletter…

• Exploring neural networks. Since I don’t have a kid but like hiking and math, here’s my version of a family Christmas card. These images walk through the inner mechanics of a neural network, visualizing what different components of the network are doing in a complex algorithm. Deep neural networks combine layers of artificial “neurons” that apply transformations to data. It starts with our input data. In our case, it’s a bunch of numerical values representing pixels in an image. The network takes that input and passes it through a series of layers where the output from the previous layer is the input to the next layer (roughly). Each layer transforms its inputs using what it has learned about the dataset so far. After going through many of these layers, it becomes possible to identify complex objects. You can see how earlier layers are identifying basic strokes and shading while later layers are showing, well, a lot of animal faces. That’s because this network was originally trained on a bunch of animal images. Here’s a bit more detail if you’re interested.

• Flipping stress upside down. Is stress bad? Well, it depends. Normally we think of stress as bad, and this has been the prevailing logic for decades. One study changed this. It took about 30k participants and observed them over 8 years in order to see how their stress and their perception of stress affected their risk of premature death. The findings? Having high stress and a perception that stress is unhealthy combined was associated with a 43% increase in premature death. However, those with high stress and the belief that stress was healthy had lower mortality rates than those with low stress! This points to the idea that mindset really matters. The researchers go so far as to claim that the mindset that stress in unhealthy is associated with 20k US deaths a year, the 15th highest cause of death above both AIDs and homicide.
• State of AI. The artificial intelligence research organization OpenAI released a new pretrained model known as GPT-3. Pretrained models allow you to reuse the findings of one AI model on new use cases. This one absolutely dwarfs the complexity of previous models. It’s a natural language model used to generate human-like text. It cost an estimated $10M for the compute resources to train it alone. Here’s a comparison of the complexity of this new model when compared to other state-of-the-art models as quantified by the number of “parameters,” which is basically the individual units of information the model is learning. For the full geeky rundown, see this State of AI Report. This means anybody can use reuse this best-in-class natural language producing algorithm without racking up a $10M bill. It’s truly mind-boggling how open sourcing this type of solution advances industries.

• High impact career. You have 80k hours in your career. How do you use them to have the most impact? That’s the idea behind the startup 80k Hours. It’s career guidance that optimizes for impact. Should you create a startup that addresses the world’s toughest problems? Or make a bunch of money at a corporation and donate it? I’ve used this site many times in my career to work through these ideas.
• Being an information juggernaut. The firehose of information we receive each day is somewhere around 174 newspapers worth a day. Oh, and that study was from a decade ago. Compare that with just 2.5 pages worth back in 1986. So how do you navigate so much information? My favorite strategy has a few tenants:
• Prioritize books because they have the greatest density of information (think of how long a blogger takes to write an article when compared with a book)
• Your attention is one of the most important things in the world since all your experience is really just the summation of things you paid attention to. Make sure the author earns that attention. Be an unapologetic skimmer
• Take note of every book that’s recommended to you and at least read the summary. If the summary entices you, read the table of contents. And if that earns more of your time, read a few pages of the introduction
• Still not sure if it’s worth the read? Try a service like Blinkist that gives you a TED-talk length, 15 minute summary of the book. Then decide whether it’s worth the time

And that’s a wrap! Hope this didn’t suck. Be sure to let me know what you think about the topics and format.

Conor