Machine Learning and the Brain

4 minute read.

Simplicity can generate vast complexity - the Mandelbrot fractal is a wonderful example. Iterating one tiny equation leads to patterns of infinite depth and detail, nearly repeating, but not quite. Computers and software are based on nothing more than the transistor logic gate and bits memory - Alan Turing realized stringing enough of them together could solve nearly any algorithmic problem. The human genome has 2.5 billion base-pairs of DNA, which is only about 600MB of data. Somehow all the information that generated you or me could fit on one CD-ROM from the '90s, even though we largely can't even fathom how our own bodies function. 

Deep neural networks are a relatively simple construct that led directly to the explosion of machine learning (ML) we're witnessing today, from facial recognition to real-time translation to image generation and artificial intelligence. They are a recursive statistical process, inspired by a theory of how the brain might work. Their promise lay dormant for decades until computers got fast enough to run them at scale. Now even the experts who create them cannot predict exactly what they will generate, as the complexity created by these simple processes is beyond our capacity to fully understand. 

I find it particularly interesting that ML algorithms suffer from many of the same foibles as people: What's the difference between ML hallucinating a wrong answer and a person bullshitting? Why are we both fooled by disguises? Why do we both stereotype and suffer from unconscious bias? How about plagiarism? The recent pile of lawsuits about ML violating copyrights shouldn't have surprised any teachers: just like students, ML can and must be taught not to plagiarize. Still, a detail of this copyright fiasco caught my attention in particular.

Image generators are now in trouble for creating very suitable likenesses of famous actors playing famous roles. This is interesting because the goal of the software is creativity, generating novel images, while what it has accomplished is also memory of its training dataset. Just from a compression standpoint it is quite a feat to be able to memorize so many images in such detail with only a few billion parameters, perhaps tens of gigabytes of memory. More amazing still since memorization wasn't even the goal - the system is in fact also quite capable of generating an avalanche of novel content.

What really impresses me about this error is that creativity and memory are completely conflated. This reminds me even more of the human brain. Have you ever written or said something and been unsure whether you were repeating someone else? How many times have you shared a memory with someone only to realize your memory has drifted from the facts? 

I've always believed the brain is just a squishy computer. An incredibly power-efficient one, sure, but a physical process nonetheless: If you physically damage your brain, it doesn't work as well, if at all. We don't understand how our own brain works, but might we learn about it by developing machine learning? Simplicity tends to underly universality - could it be that the simple recursive statistics we've developed that generate unpredictable complexity might also describe the simple biological circuits that give rise to our own thoughts?

Imagine for a moment this is true; imagine a future where our software can accurately mimic human thought. What would it mean about us if our own minds can be synthesized by a deterministic computer? Might that mean our own minds are no less algorithmic? Would that imply free will does not exist?

I think the real problem with this line of reasoning is that free will is often presented as a dichotomy between determinism and choice. I think that what matters instead is predictability. Humans walk a fine line on predictability: not predictable enough and you're crazy; too predictable and you're a drone. We categorize processes as deterministic or random, but what do we really mean? 

Random means we can't predict the next step. Some computers contain a chip to sample numbers from a noisy thermal process to ensure true randomness. A pseudo-random number generator is a completely deterministic algorithm, yet to one who doesn't know the algorithm, is its output really any different? Chaos theory describes mathematically how deterministic physical processes become unpredictable, because the sensitivity to error hits a singularity where no measurement is precise enough to make a useful prediction. Many physical processes are chaotic - fluid mixing is an example. 

So what then is choice or free will? Certainly our choices are influenced by our circumstances, our inputs. Yet we are not completely predictable. And even if we manage to simulate our own brains very accurately, I have no doubt we'll find the system has enough chaos to make it still unpredictable even if we were to gain full knowledge of how it functions. If you want to say this means free will is an illusion, fine, but practically speaking it changes nothing because there are still fundamental limits on what we can predict about ourselves or each other. 

I believe the process of developing artificial intelligence will lead to learning as much about teaching ourselves as teaching machines. We must fight plagiarism - by both humans and machines. We must fight stereotypes and unconscious bias - in both people and algorithms. We must teach creative thinking - to students and AI. 

We must learn to fix a damaged brain, as well as build an efficient computer. Perhaps we will take inspiration from the brain and develop analogue computing architectures for machine learning. After all, it is a statistical process based not on precision, but on approximation and randomness. Perhaps the determinism of binary logic is actually a hinderance for these algorithms. Perhaps computing and biology are not so different in the end.