Never did We send a messenger or a prophet before thee, but, when he framed a desire, Satan threw some (vanity) into his desire: but Allah will cancel anything (vain) that Satan throws in, and Allah will confirm (and establish) His Signs: for Allah is full of Knowledge and Wisdom. (Quran 22:52)
That He may make the suggestions thrown in by Satan, but a trial for those in whose hearts is a disease and who are hardened of heart: verily the wrong-doers are in a schism far (from the Truth). (Quran 22:53)
Axl Rose at The Whisky, 1986.
As far as its neighbors are concerned, a neuron can only be in one of two states: firing or not firing. This misses an important subtlety, however. A typical neuron spikes occasionally in the absence of stimulation, spikes more and more frequently as stimulation builds up, and saturates at the fastest spiking rate it can muster, beyond which increased stimulation has no effect. Rather than a logic gate, a neuron is more like a voltage-to-frequency converter. The curve of frequency as a function of voltage looks like an elongated S and it is variously known as the logistic, sigmoid, or S curve. Peruse it closely, because it’s the most important curve in the world. At first the output increases slowly with the input, so slowly it seems constant. Then it starts to change faster, then very fast, then slower and slower until it becomes almost constant again. The transfer curve of a transistor, which relates its input and output voltages, is also an S curve. So both computers and the brain are filled with S curves. But it doesn’t end there. The S curve is the shape of phase transitions of all kinds: the probability of an electron flipping its spin as a function of the applied field, the magnetization of iron, an ion channel opening in a cell, water evaporating, the inflationary expansion of the early universe, the spread of new technologies, white flight from multiethnic neighborhoods, epidemics, revolutions, and much more. The Tipping Point could equally well (if less appealingly) be entitled The S Curve. Joseph Schumpeter said that the economy evolves by cracks and leaps: S curves are the shape of creative destruction. The effect of financial gains and losses on your happiness follows an S curve, so don’t sweat the big stuff. In Hemingway’s The Sun Also Rises, when Mike Campbell is asked how he went bankrupt, he replies, “Two ways. Gradually and then suddenly.” That’s the essence of an S curve.
The S curve is not just important as a model in its own right; it’s also the jack-of-all-trades of mathematics. If you zoom in on its midsection, it approximates a straight line. Many phenomena we think of as linear are in fact S curves, because nothing can grow without limit. Because of relativity, and contra Newton, acceleration does not increase linearly with force, but follows an S curve centered at zero. If you zoom out from an S curve, it approximates a step function, with the output suddenly changing from zero to one at the threshold. So depending on the input voltages, the same curve represents the workings of a transistor in both digital computers and analog devices like amplifiers and radio tuners. The early part of an S curve is effectively an exponential, and near the saturation point it approximates exponential decay. When someone talks about exponential growth, ask yourself: How soon will it turn into an S curve? When will the population bomb peter out, Moore’s law lose steam, or the singularity fail to happen? Differentiate an S curve and you get a bell curve: slow, fast, slow becomes low, high, low. Add a succession of staggered upward and downward S curves, and you get something close to a sine wave. Children’s learning is not a steady improvement but an accumulation of S curves. So is technological change.
The Master Algorithm (2015) by Pedro Domingos
The primary purpose of your brain is to keep you alive. Our brains focus on 3 simple motivations to increase our odds of survival: avoid pain, seek pleasure and conserve energy. This is the motivational triad. You will always be motivated to do something that won’t hurt you, feels good, and is easy. This is how we humans survived back in the day. We were motivated to hunt, have sex and seek warm shelter by our desire for pleasure. We stayed alert to avoid danger. We didn’t go jogging just for the sport of it. We conserved our energy.