The word is not enough

Computers have their 0s and 1s. What do we have?

What if I told you that the language you are most familiar with, grew up speaking, the one you think in, is a foreign language you have forced yourself to learn. Computers have their 0s and 1s, what do we have?

In his conversation with Mark Zuckerberg (jump to 1:35:20), Lex Fridman asked a question that has stayed with me and prompted this post: can AI improve the communication between two people who speak the same language? Can it outperform both their abilities with the language and capture the chemistry through translation? AI translation today is great for helping a non-speaker converse in another language. The more difficult test is whether AI can help speakers of the same language communicate better with each other.

In that question lies an uncomfortable void in humans: we have no “binary” format in which to communicate with other human beings. Putting thoughts and feelings into words is at best the JSON equivalent; there is no Protobufs. Perhaps this is why the feeling of “presence”, of being in the same room with someone, cannot be serialized over the wire — it is why Zoom meetings fail at letting people build connection and quickly become tedious. Perhaps ten thousand years ago and a rung or two lower down in Maslow’s hierarchy we did not need as much of an ability to communicate — a few words spoken out of the mouth and into someone else’s ear was sufficient for survival, and for co-operation or competition within one’s tribe, where the tribe provided enough shared context.

The potential for technology to change this could redefine the human experience. Imagine being able to, very quickly and effortlessly and accurately, communicate everything — everything — you are perceiving, thinking, and feeling to someone else. What would that mean on a date, in an argument, in a commencement speech, when reporting from the front-lines of a war, in writing a journal to leave behind for your children…

Simple examples of modes of “binary” communication are easily found around us, and getting more sophisticated by the day.

Dall-E 2

Dall-E 2 is a new AI system from OpenAI that creates realistic images from a textual description. It took the tech community by storm when it launched, and with good reason. The results are often fantastical, always unexpected and perhaps even more creative than a human’s rendition. Take a look at some of the popular examples:

Critter @BecomingCritter

teddy bears working on new AI research on the moon in the 1980s

Critter @BecomingCritter

A human basking in the sun of AGI utopia

Arpit Ingle @arpitingle

DALL-E 2 result for "Early drawings of the iPhone by Leonardo da Vinci"

These are not simply representations of the words: the imagery makes the words come to life, carry and convey emotion. It is not a leap of imagination to expect Dall-E 2 to be able to generate very precise images carrying exactly what you are feeling right now, that would be quickly and instinctively understood by someone else. ‘Emotion transference’ is what I call it. We already use a lossy form of it through analogy — “butterflies in my stomach”, for example.

This tweet (and replies to it) reinforce my point:

Sam Altman @sama

i have never been a visual thinker, but using dall-e seems to have slightly rewired my brain. i can sort of do it now!

More Dall-E 2 images in this thread; click through if you’re interested:

Critter @BecomingCritter

Thread of DALL-E AI generated images and their prompts

Google Search

Google came to a similar conclusion that there are things beyond what can be easily expressed in words, and are trying a new feature where users can search using images: https://www.cnn.com/2022/04/07/tech/google-lens-search-pictures-words/index.html

We had a similar realization at Twitter when we were trying to categorize the universe of content on Twitter: it was impossible to label many categories because they simply defied description. A good example is the subreddit r/keming: it is really hard to explain what it is about or why it is called ‘keming’ (the word is not in the dictionary) unless you see the images posted there.

Emojis

Emojis are low-tech compared to something like Dall-E, but let me just drop a few emojis here to make my point on what a succinct and rich vocabulary they provide: 🤷‍♂️ 🤦‍♂️ 😤 💀

Notice how that last one does not refer to danger literally (quite the opposite!), and may even communicate how clued in one is to online culture.

Gifs & Memes

These are similar to emojis, but with even more cultural cachet inbuilt. It has become an art and a game to be able to respond with only a gif or a meme. You are either playing that game or taking the blue pill. KnowYourMeme.com is a great website cataloging popular memes and explaining the context behind each, and you can spend hours on it understanding popular culture.

Ted Chiang, winner of four Hugo awards, four Nebula awards, four Locus awards (you get the picture), in his 3-page short story ‘The Evolution of Human Science’ also plays on this theme:

No doubt many of our subscribers remember reading papers whose authors were the first individuals ever to obtain the results they described. But as metahumans began to dominate experimental research, they increasingly made their findings available only via DNT (digital neural transfer), leaving journals to publish secondhand accounts translated into human language. Without DNT humans could not fully grasp prior developments nor effectively utilize the new tools needed to conduct research, while metahumans continued to improve DNT and rely on it even more. Journals for human audiences were reduced to vehicles of popularization, and poor ones at that, as even the most brilliant humans found themselves puzzled by translations of the latest findings.

There are no meta-humans in our world, but the efficiency of language is a handicap. Using language as the primary means of communication is like walking as the primary means of transportation. We have barely scratched the surface of what technology can do to make the human existence more … human.

We are getting closer to finding our “binary” language. It is going to be an epic journey of technology interwoven with culture.