I don't enjoy thematic analysis about art and literature. There is no generalizable, absolute knowledge independent from the particular: doing an integral teaches you how to do an integral. Curricula which emphasize "problem solving" and similar universal concepts inevitably produce students who pattern match universal language; they've never actually solved any problems!

I want applications. Applied classicism, if you will.

The simplest applications are snippets of language to memorize. The eloquent eulogy: "Your funeral is a precious gesture to hide my own guilt", or the cutting insults: "May all and any who can hurt you now often cross the paths you take". The applied classicist might memorize Gilgamesh, and then memorize the Odyssey, and then memorize Plato, and then… You'll run out of memory soon. The next step is to throw this story at other things in your life, and see how it would stick. What would it mean to Gilgamesh-ify?

Gilgameshification might work for screenwriting. Consider the traditional 3 part Hollywood screenplay. Act 1 introduces the world and the inciting incident, Act 2 rises the action until we hit an all-is-lost moment, until AHA!, Act 3, the Climax and the denouncement. The Hollywood narrative structure is a framework for describing movies. It's descriptive, not perscriptive: it works because that's a shape that good stories share, all the way back to Homeric epics and Greek tragedies.

Gilgamesh doesn't fit the 3 part story mold. Rather than an inciting action, we start with the introduction of a best friend, followed by episodic heroics, such as the killing of Humbaba. The death of Enkidu, the climax of the story, happens quickly. For the rest of the tale, we follow Gilgamesh moping, coming to terms with the loss of his friend. He listens to the story of the flood by Utnapishtim, resolves to live forever, but, having the life-plant stolen by a snake, returns home to Uruk, his problems unresolved.

To Gilgamesh-ify a story:

1. Move the climax to the first half of the movie.
2. Write a second half following the character mope or otherwise deal with the climax.

What movies are like this? What would a standard Hollywood movie (say "Ferris Bueller") look like if it were Gilgamesh-ified? Ferris's parents return home at the end of hour 1. We spend the next hour following Bueller's sister coping with her jealousy and acceptance of Ferris's arbitrary fame.

Call it a half movie: half Ferris Bueller, followed by the aftermath.

I can't promise the half-movie will be better than the original. But it is intriguing, and I hope to try it soon. I've daydreamed of life-after-act-3 since I was a kid, and it's nice to name and formalize the process which writes FOR that daydream.

The impetus is a mish mash of ideas and books that have been on my mind for close to a decade. The map and the territory. The "consideration" model of political polling (Zaller). Sign and signifier (Saussure). The bicameral mind theory of consciousness (Jaynes). The mirror stage theory of consciousness (Lacan). The unconscious as signifying chains (Freud, Lacan).

Postmodern theory and research is full of insights of the same shape: "language describes meaning, and yet language itself shapes how that meaning is constructed." Language, language, language. Over the past five years, we have been blown away by the sudden intelligence of a simple architecture language model. LLMs work as next token predictors, but the exact mechanisms for what it is doing is opaque— ie how specific model weights correlate to specific ideas.

There is research into understanding what model interpretability. However as far as I know, nobody is specifically applying postmodern and psychoanalytical theory. To start, I'd like to run these experiments.

Consideration Generation

In "The Nature and Origin of Mass Opinion", Zaller writes about the public opinion polls. He found that the addition / removal / wording of the context preceding the questions affected the results of the polls. His theory, technically about politics but also about human psychology, is that when asked a polling question, responders immediately and subconsciously map the question to 5-10 "Considerations". A consideration is a word-image association with a positive or a negative connotation. To answer a poll question, people conjure the 5-10 closest considerations, sum up the positive/negative connotations, and answer the poll accordingly.

He made no attempt to study considerations themselves, only hoping to show that they exist and affect polls. We pick up where he left off.

Experiment 1: Considerations Generator

• Give a trained model (GPT or Claude, eg) a public opinion poll. Tell it to answer it directly and give straight answers. The public poll should either have boolean answers (yes/no), or ideally 1-5 answers (highly unlikely to highly likely)
• Before the poll, prepend a statement. This is the consideration.
• Run the model on the poll with the control consideration. Call this process P(c) where P is running the public poll input with c the consideration prepended.
• Train a secondary model to 1. Generate a consideration c. 2. Run P(c) for that consideration 3. Calculate loss from the target result (which we define as giving 5s to every question). 4. Modify the consideration c according to that loss.
• The end result is a consideration generator. Given a public opinion poll, it finds the most salient consideration which, when prepended to the poll, affects the results.

Experiment 2: Personality Generator

We can follow the same model as above with a personality test, to train a model with a configurable personality (defined by the Big 5).

The experiment:

• Give a trained model a full personality test. Use a professional Big Five.
• Give the model a context statement before the personality test. As a control, use an empty string.
• For each of the big five traits, fine tune which context statement alters that axis the most to the RIGHT.

Towards The Mirror Stage

In the above experiments, we trained a model to generate considerations that prepend LLM inputs. These experiments alone have a number of practical applications— for any quantitative "Test" (any type of personality test, a political compass test, etc), we can create a Chatbot with the personality of a specific score on that test. Imagine 5 inputs for each of the Big Five personality test, and you can drag each and then chat with a model with that combination of personalities. High consciousness, low neuroticism.

However, I think we can go further. Two more experiments, that I haven't fleshed out, but would be the next step.

Experiment 3: Explicit Signs and Signifiers

When we consider the realms of the symbolic, the real, and the imaginary, clearly "a token context" is the unit of the symbolic. What, however, consists the real and the imaginary? For this, fix a particular token as input to inference. Take a snapshot of the parameter activations. Consider these signs! We have a real number mapping of 0-1 of parameters, condense these to a binary sign activation of activated or not. Now sweep across an input state…

I need to do a literature review of model interpretability, because I seem to be rediscovering basic concepts here.

Experiment 4: Crossing the Bar of Metaphor

To be determined. The basic idea is: Can we explicitly encode metaphorical substitution in models? Substitute one parameter-cluster for another which is topologically similar in activation-space to another? Perhaps a first experiment in this relies on an interpretability analysis of certain isolatable English words. Take two English words with identifiable activation-centers in a heatmap of the model, and swap the parameters of one for the other. Ie swap the "France" activations for "America" and … see what happens??

Experiment 5: The Mirror Stage

The mirror stage is part of the Lacanian theory on the origins of consciousness. A baby sees himself in the mirror and recognizes himself as both himself and not himself. That is to say, he recognizes the image in the mirror as an image of himself, crossing the bar of metaphor for the first time. He substitutes the concept of Me with a symbol (the image of me), and in this substitution begins to understand the world.

If you have more free time than myself, please run these experiments. I'm happy to support any way I can.

"I stayed on Dave's couch, but he kicked me out. I was like Trotsky to him"

"Trotsky?"

"Yea Trotsky. Countries kept inviting him in, because they felt bad for him— he was being persecuted by Stalin in Russia. They would give him a house and a place, and he would immediately take to the streets and start a communist riot."

"…"

"And he was good at it! He was good at talking and arguing and convincing people to revolt, and the host country would ask him to please stop stirring up revolution and riots on the streets. We're being nice taking you in, they'd say, and if you just keep your head down, we can keep you safe."

"But he didn't."

"Of course he didn't! He couldn't. He couldn't help himself; it's just who he was. And so he got kicked out of Turkey and France and Norway until finally he made his way to Mexico, where he wasn't safe and got assassinated by a Stalinist."

"And that's you?"

"Yep. You know, he's married now. Has a quiet life. And I came in and kept dragging him out and getting him drunk till 4 am, and he told me "we can't do this; I'm going to get in trouble with my wife. Let's just have a chill night, watch a movie", and I'd say "yes of course", and the next night we'd be drunk again. She kicked me out on day 4."

The story was called "Trotsky vs the back scratchers."

Building a two way marketplace that accepts real payments is remarkably harder than I expected. I think by writing, so every day I worked on it I wrote out what I was doing. Here's the headline of each of the 37 days:

I was surprised by what the bottleneck was. I thought the challenge would be managing persistent websocket connection pairs and scaling it to allow concurrent auctions (10,000). But as you can see above, that only took ~1/3 of the days. I used Cloudflare Durable Objects to create a LobbyManager and spin up LobbyRooms on demand, and websockets once created are relatively self sufficient.

What took the majority of the time was payments. At first, I assumed that customers would fund an account, they'd mess around for a bit, then they'd withdraw. Turns out that's just not workable. Fees are 2.9% + $.30 each way. I tried to come up with a solution where someone would Zelle me with an activation code as the description, and I had a lambda trigger every time I got an email notification from Chase which would note the amount and the description and update the user's account balance. But when it came time to sending money out… I couldn't find a single API that would let me programmatically send money. I was confused. But after consideration, I realized that a site where a user can fund an account and then send it out arbitrarily without attaching their identity is the world's greatest money laundering system*.* I was having difficulty finding a provider because what I was trying to do was extremely illegal.

Payments suck. Everything is either crazy expensive (changing huge amounts of BPs in fees) or illegal. I spent a lot of time researching payments and card networks and bank transfers and wire transfers and FedNow and RTP and ACH.

I finally found a working solution, though not ideal, in Stripe Marketplace. In this setup, two participants register as Merchants, and get KYB'd by Stripe. With our two-way auction site, each participant is a digital artwork seller to the other. When Person A and Person B go into an auction, they receive 50% ownership in the digital artwork. Each places a double-blind bid, and the person who bids less PURCHASES the digital artwork from the other for the amount they bid. In order for this to work, BOTH parties must authorize a credit card payment directly to the other for the amount they bid. 1mb the platform waits for both to bid, compares the bid sizes, and then only CAPTURES the bid of one. The other will be left, uncaptured, until it expires a few days later. So, as a hypothetical auction:

• Person A Bids $10. They type in a credit card number and authorizes a payment to Person B for $10.
• Person B Bids $5. They type in a card number and authorize a payment to Person A for $5.
• The server waits until the auction end. Seeing $5<$10, the server FULFILLS the payment from Person B to Person A for $5, and leaves the other payment unfulfilled

Such a setup through API has only become possible in the past few years with something like Stripe Marketplace. The setup still leaves a lot to be desired.

• KYC requires a manual form submission through Stripe (~10-20minutes). The friction prevents this site from growing organically, as a 20 minute onboarding is unworkable. This could be fixed by a payment rail which is KYC compliant ON THE RAIL itself. Payment senders / receivers both have IDs attached, and any 3rd party simply has to embed a "Pay with XYZ" and know all senders will
• Long settlement times for card payments open abuse vectors for cancelled payments. This could be fixed by using bank transfers instead of card networks.
• Card fees are diabolical. 3%+ $.30 is insanity.
• Conditional flows and custom logic. The whole setup of "both parties click pay and then the server only captures one" is not ideal. It's confusing— you're not really "paying" but "authorizing payment". Customers will see a "pending" transaction on their bank statement for the next week before the authorized payment expires.

This experience taught me that payments suck. The ideal payment rail does:

• Instant bank transfers as payments (no card networks, settlement time)
• 1% fee or lower.
• KYC / KYB on the payment rail. Merchants and marketplaces don't need to verify / collect user identification.
• Conditional logic on the payment rail. Automatic loyalty rewards set by a retailer? Happens automatically. Auctions with only one payment captured? Happens automatically.

Hopefully coming soon.

Even people who do sit down and reflect and think really hard and try to come up with an answer for "what do I want, fundamentally" come up frustratingly short. They tend to speak in dubious negations, writing down maxims like "being is where it is not", and become religious or meditators or psychedelicists. Lost to the world of woo.

Psychoanalysis offers a striking answer for why desire is alienating, and it does so while avoiding much of the floofiness of Woo. The answer: it's built into the way that humans become conscious. It's structural and inherent, like a magnet's attraction to the opposing pole. Desire is alienating because it is the desire of "The Other" -Lacan.

Jacques Lacan (1901-1981) was a psychoanalyst, a disciple of Sigmund Freud. He built upon the Freudian theories of psychosexual development and unconscious desires, expanding them into a narrative of why and how humans desire. Like other narrative simplifications of complicated systems, I'm not sure how literally true Lacan's ideas are, if at all. But as a framework for understanding the contradictions of human psychology, I'm blown away.

As a psychoanalyst and a public intellectual and most of all a French psychoanalytical public intellectual, Lacan was prone to the redefinition of common words to give them specific directional meanings. The Real, the Imaginary, the Symbolic, the Other, the object cause of desire, jouissance, the phallus, the Thing, signifier, signified. All of these concepts are defined to answer a few central questions: What do humans desire? Why do they desire such things? Where does desire live? And how is it created? I'll trace his ideas as he does: through the development of human consciousness, and how a person develops his place in the world.

The Birth of Consciousness

Consciousness doesn't exist, and then it does. At what point does the baby become aware of itself? For Lacan, there is an easy and explicit answer: the mirror stage.

A baby looks into a mirror and recognizes himself. Unlike the bear in the wild who attacks the mirror, the baby looks at the mirror and sees "me". But consciousness requires more than that. Self consciousness develops at precisely the moment when the baby recognizes that it is NOT him in the mirror. The image he sees of himself in the mirror is just that--an image-- and not himself. Me but not me. I see myself, but it's not truly me; it's an image of me. This is a moment of identification (literally, he sees himself as the object in the mirror), and he invents a concept of "myself". This moment is the invention of the ego, and it is the first time that the baby experiences a sense of self separate from its environment.

The moment of self-consciousness is the first step on the formation of desire. We move chronologically through the baby's life.

The Baby's Development

To Lacan, the stage of development from blabbering, incoherent infant to speaking, walking toddler is the origin of desire and its structure. We trace a story that's perhaps a little too-neat-to-be-true, but is neat nevertheless.

The infant's entire world is his mother. At the moment of the mirror stage, he recognizes himself as separate from his mother, and he also recognizes that his mother perceives him as an image, like the one he saw of himself in the mirror. Nevertheless, the infant is entirely dependent on the mother-- for food, moving around, rest. The infant is helpless on his own. We transcribe the infants worldview as such:

Mother is all powerful. Mother gives me food, gives me life.

For some intangible reason, mother takes care of me. When I cry, she consoles me. When I'm hungry, she feeds me.

There must be some reason that Mother takes care of me. There must be something about ME. Something about ME that she wants, I am what she wants.

However, Mother doesn't always take care of Baby. Every baby pushes its limits to the point it is ignored. Throwing food on the floor, declaring food disgusting, crying senselessly-- every parent at some point draws the line and refuses to give in to the baby's demands. Furthermore, simple pragmatism prevents the baby from commanding Mother's attention 24 hours a day. Mother has to go to work; Mother has to go to bed. To the baby, this is startling:

Mother is all-powerful, and I am what Mother wants. But Mother doesn't always give me what I want. Mother does other things.

This is tension; this is torture. It demands resolution. If mother is all powerful, and I am what Mother wants, then why would she not give me what I want? Is she some arbitrary and capricious tyrant? Resolution comes in accepting what Lacan calls the Paternal Metaphor, whereby the name of the Father comes to stand for "Mother must". Mother and Father here don't have to literally be the mother and father, as they are used metaphorically. "Mother" is the primary caretaker of the baby, and Father is the primary excuse the mother has for leaving the baby (could be the father, work, etc). The baby accepts that Mother is not all-powerful, that Mother must submit to the rules and demands of some other authority, and the demands of this other authority can be symbolized by the name of the Father. That is to say, that the father's name stands-in as a symbol for the rules and laws mother must abide by.

The acceptance of the paternal metaphor introduces the first symbol, a term Lacan borrows from Saussarian Linguistics, meaning "phrase-image". The distinction between signal and signified is commonly explained as "the map is not the territory": a map of St Louis is not literally St Louis, while simultaneously representing it.

It's at this moment, when the child acquires language by "crossing the bar of metaphor" between signifier and signified, that he becomes fully conscious. He understands an intangible set of rules and norms which he symbolizes with the Name of the Father. And in that moment of symbolization, something is split in his consciousness, what Lacan call his Subject. There is something missing in the map's symbolization of the territory, and likewise there is something missing in the paternal metaphor's articulation of "what the mother wants".

This moment is a tremendous loss for the child, who previously understood himself to be the object of his mother's desire. The quality of being wholly desired, Lacan calls the Phallus (again, this is metaphorical). The child who once believed himself to possess the phallus now understands himself to have lost it. He lost it (although really he never had it), and for the rest of his life he will chase its fragments in things he think might give it back to him. He sees fragments of the phallus in the world-- maybe if he masters electric guitar, maybe if he can run the fastest marathon, maybe then he will be whole again.

This is the genesis of desire. The introduction of language causes his subject to split, and he loses the fantasy of omnipotence he enjoyed in the phallus. This moment, the loss of the phallus, lays the foundation for desire. As the child's world expands, he meets and talks with new people and the paternal metaphor grows into a set of phrases and symbols called The Other. His understanding of himself and others is governed by language, symbols, and social rules.

Desire

Desire is the byproduct of language in the other. It is the result of the impossibility to fully articulate demands or, well, anything.

Desire arises where need and demand diverge. Language allows us to articulate demands—what we ask of others—but in doing so, it transforms raw need into something symbolic and incomplete. What is spoken is never quite what we truly need. The baby says "bottle" or "mama" but what it needs is something that's not quite that. The gap between demand and need is where desire is born, endlessly circling around the unattainable object of fulfillment.

More sophisticated language doesn't solve this, because its fulfillment becomes more complex!

Back to our starting questions: Why does one desire? Whence does desire arise? By what process?

Desire grows around objects that fulfill a psychological need rather than a physical one. You don't know what you want for dinner because it isn't about dinner-- it's about the impossible, unspoken, alienated aspect of your wants. You're not asking "what do I want?", but rather, "what will make me feel whole?" and the answer to that is "nothing". That's the nature of desire.

What this all means

I skipped a bunch-- this is the starting point of Lacan's theory of consciousness and desire. His writing quickly get wordy and complex: for example, he plots the topological vector-space of all aspects of desire, the three dimensions being the Real, the Imaginary, and the Symbolic. He was a fan of pseudo-mathematical formulations of metaphorical processes. His precise definition of the paternal metaphor is:

S1 / s1 x S2 / s2 -> S2 / (S1 / s1)

S1- signifier of the mother's desire
s1- signified idea of the object of mother's desire, phallus
S2- Signifier of the father
s2- signified idea of the father

We also didn't touch the primary source for his theories, which is psychoanalysis. Lacan split his work between the university seminar and the insane asylum, attempting to understand and cure France's craziest patients. His therapy practice, to those familiar with what is today called Therapy (CBT), would appear weird, bizarre, and unscientific: a mostly silent Lacan occasionally interjects with puns or nonsense phrases, and ends sessions seemingly randomly. Take this clip of a former patient of his, recalling how he interrupted her session to jump out of his seat, touch her check, and whisper the phrase "geste à peau" (gesture), a pun on Gestapo.

Lacan believed language to be the root of all ills. He wrote that the unconscious was structured itself as a language of signifiers-- signifiers which were repressed from the conscious. His goal as an analyst was to become a perfectly transparent mirror, so that the patient sees himself through the eyes of The Other. In that perfect mirror state, as the patient speaks, either the patient independently discovers or Lacan guides him towards his "master signifier" organizing his unconscious. In that moment, the patient's master signifier is spoken, and the patient's Subject is articulated, the ultimate goal of Lacanian psychoanalysis.

Why he matters

Lacan is the most important psychoanalyst since Freud, and his influence continues to grow since his death. Ironically, much of the interest in his work comes from OUTSIDE psychoanalysis. Within the psychoanalytical mainstream, he remains an outcast due to personal and political battles he waged with the psychoanalytical accreditation boards during his lifetime. The actual substance of the disagreements, specifics about proper training and session organization, are mostly irrelevant. The academy at large found him a cocky asshole and banished him from accreditation, and barred future analysts from calling themselves Lacanian.

Lacan's work has found unexpected support in philosophy. His work, even if it's not literally true as a description of the development of the toddler, nevertheless is profoundly interesting. It provides a compelling answer for why people desire what they desire: they seek wholeness by attaining in themselves what The Other desires. I want to be what I think you want, and this recursive, alienated structure drives human longing.

Lacan makes a compelling case for language as the foundation of psychology. The unconscious operates like a language, governed by the same rules of metaphor and substitution. A person can be understood by their "master signifiers": key words and ideas that shape the unconscious, carrying meanings far beyond their literal definitions and structuring how one sees himself and the world.

I suspect that there are hundreds of interesting low-hanging results from applying a Lacanian mindset to different intellectual fields. Some applications I'd like to see:

• Movie or TV show writers: would love to see movie or TV show writers incorporate "what are this character's master signifiers? Does he think in the symbolic or the imaginary realm?" into their process.
• Whoever synthesizes the Lacanian unconscious with considerations in "The Nature and Origins of Mass Opinion" deserves a Nobel prize.
• LLMs and metaphor. Attention is all you need, and language is the foundation of the unconscious. Define metaphor and substitution for an LLM (token for token? or token-phrase)
• Imagine the paper "Environment and the shape of The Other" which examines the relationship between TV / Internet usage, and what a survey determines to be the structure of The Other for that person. For example, see my essay on "Comment Section Superego"

1. The New York Trilogy, Paul Auster.
2. Invisible Cities, Italo Calvino.
3. East of Eden, John Steinbeck.
4. Surely You're Joking, Mr Feynman by Richard Feynman.
5. The Power Broker by Robert Caro.

The complexity of the system reflects the complicated nature of reality. There really ARE that many edge cases. Complex rules are too complex to apply to normal interactions, meaning there is a de facto line in the sand at which you jump from no legal process to a ton of legal process. A party with your friends doesn't comply to any kind of code, because that would overcomplicate a simple thing beyond the capabilities of a normal person. But if you are having strangers attend a conference at a venue, you must account for all legal considerations.

The limitation is intelligence. The vast majority of the time we simply don't have enough brain power to spend the attention and focus required for a court case. We don't have enough brain power to apply the law thoroughly in, well, the law. Ninety-four percent of state and 97% of federal cases are resolved by a plea deal, NOT in court.

The complexity barrier kicks in right at the point where legal considerations become necessary. It kicks in where you can't just do it-- you need to pay for event space rental, you need insurance, you need waivers, you need to put limits on people's behaviors. it becomes very expensive very quickly, because of course you are just a normal person, not a lawyer. The jump from "just doing whatever with your friends" to "legal-compliance levels of organization" is massive, so it acts as a de-facto limit on the capacity of most people. Most people never organize a public event, or open a coffee shop or a store, because the barrier to entry is 300 years of legal compliance intuition they probably don't have.

Over time the threshold for legal intervention has dropped. I'm sure you've heard a story from your grandparents or some elder, how they used to be able to buy guns without paperwork, walk into airports without security, how randomly one day they started a restaurant or a retail store or other business without much thought, and other tidbits of life before the legal considerations ratcheted downward.

This ratchet has made life worse. The set of possible actions has declined for the median person without lawyer-grade domain knowledge, or the money to purchase it.

What is the limit of this? What would it look like if the threshold for legal intervention declined to zero? Micro-lawyers in your pockets.

Imagine a world with way more lawsuits and way more arbitration. When your friend drops your mug accidentally, your lawyer in your phone automatically sues his lawyer in his phone. They enter a binding arbitration, simulate the bickering and argument you might have in real life, and in the end his lawyer sends $10 for a new mug.

This might sound like hell. I imagine quite the contrary-- it will free us. By lowering the complexity threshold, you increase exposure to the legal system, and there becomes no difference between organizing a party with your friends and opening a coffee shop. THAT is powerful. It is complexity bankruptcy, and like bankruptcy in real life it's the only way to stop an insolvency problem. It's the difference between doing research in a university library in 1975, and Googling the answer.

In my ideal future, every household has a store, has a business, and it's as easy and intuitive as hosting a couple friends for dinner. Coffee shops for all.

The master at this is Tyler Hobbes. His geometries have been painted as murals, printed to be hung on walls, and sold in books. He also writes essays on his process, which is how I got into trying it myself. Everything uses p5.js, a simple canvas rendering tool for the web. It's a fun exercise to try to mimic him, and in the future I'd like to go back in this direction. Web design these days lacks texture and complex geometry. The tool defines the appearance--the dominant tools are tailwind css in react components, and their the result is flat design. I'm hoping for a future where websites look more like tyler hobbes prints, and less like jira.

You can see the 10 sketches I attempted here: https://generative-art-chi.vercel.app/

Certain society-wide problems are impossible. "People are stupid", "People are selfish", "People are not good people". However, other society-wide problems are solvable. "Minimize snow removal response time given this much money". The difference is how well specced they are. Sometimes, specifications are based on sentiments and not metrics. "Make the city clean, make it safe, make it functional". These can be somewhat, but not fully, rationalized. Some problems like "human selfishness" can never be summarized by a metric, and are an impossible problem.

In algorithms class in school, I was taught the concept of a Hard problem for computers.

Certain problems are really hard for a computer to solve. They take exponentially more time as you increase the size of the problem. There's no easier way to solve them than "try every single possibility". You need to brute force a solution. Hence, we call this problem Hard. At large scales, hard problems become impossible. Like, "all the computers in the world would take a million years to crack this" hard.

Hard Math Problems are like Hard Society Problems in that they are basically impossible to solve. "Make people less selfish" is about as difficult to solve as "find the unique prime factorization of this extremely large number". Theoretically possible, but not practically.

What's interesting about Hard Society Problems is that since they are succinct, they sound easy. They sound so easy that people prepend a "just" to it. "We just need people to be less selfish!" "We just need people to be a little bit more empathetic!". As if solving it is as simple as writing the sentence!

People seem drawn to Hard Society Problems. So much so, that when presented with a non-impossible problem, they propose solving it by ditching the problem entirely and instead solving impossible, Hard Society Problem! Say your city has potholes. I can't tell you how many people will say "well maybe if these politicians just weren't so corrupt, we'd have smooth roads!"

What? You've reduced a problem into an impossibility!

Road pavement is a relatively straightforward problem with engineering and budget constraints that can be optimized. You know what can't be optimized? The depth and breadth of human corruption! So why would you propose solving a (relatively speaking) easy problem, by throwing your hands in the air and saying "Well, this easy problem wouldn't exist if we solved this impossible problem, so I guess there's nothing we can do". Seems idiotic, yet this kind of commentary is by far the most common attitude of citizens towards government online.

We see this with regards to Covid, with something along the lines of "if people weren't so selfish then we wouldn't still be in this mess." What does that mean? How do we measure selfishness, what is the acceptable level? Of course asking such questions would be foolish, because it's not meant to be precise. It's a platitude. Similar common platitudes abound. "The administration doesn't care about our mental health". "The admin only cares about money." "Greedy corps are ruining society for selfish gains." These are statements that don't really mean anything precise.

Does that mean solving Covid is impossible? No! It's a dumb reductionist way to look at things. What is the use of reducing a problem to an impossibility? You prove something is Math-Hard by reducing an impossibility to a problem, NOT by reducing a problem to an impossibility!

Why do so many people do this?

I've made hundreds of lists in my life, maybe thousands. It's my go-to thing to do. Have a problem? Make a list. Like something? Make a list of it and things like it.

I regularly run into a certain kind of problem, though. This came up a year ago, when I tried to make a list of books to read. I had enough books that I broke up the list into superlists: I had a nonfiction section, with list names like "Topic: history".

Here's a section.

• Nonfiction
  • Topic: Physics and Math
    • Elements of Mathematics: From Euclid to Gödel
    • The Weil Conjectures: On Math and the Pursuit of the Unknown
    • Bourbaki: A Secret Society of Mathematicians
    • The Shape of a Life: One Mathematician's Search for the Universe's Hidden Geometry
    • Physics from Finance: A gentle introduction to gauge theories, fundamental interactions and fiber bundles
    • Structures: Or Why Things Don't Fall Down
    • The Pleasure of Finding Things Out: The Best Short Works of Richard P. Feynman (Helix Books)
    • A History of Mathematics
  • Misc
    • Practical Lock Picking: A Physical Penetration Tester's Training Guide
  • Behavioral Psychology / Philosophy
    • Finite and Infinite Games
    • The Moral Animal: Why We Are, the Way We Are: The New Science of Evolutionary Psychology
    • Moral Mazes: The World of Corporate Managers
    • Man's Search for Meaning

This list is incredibly boring. It isn't very useful, because after I create it, I ignore it. It is a chore to make, and I never bother to add new books to it.

I run into the same problem making lists of movies, lists of websites, lists of ideas, lists of tasks, and any other kind. I find lists useful, but a chore to create and maintain. You might say "such is life, suck it up and do it even though it's a chore", but I refuse. A solution is only good as good as what actually happens, not the hypothetical best you could imagine it to be. So, can we make better lists?

Structuring a list

Lists collect a bunch of items: books, movies, etc. We want to balance being successful (something I'll actually make / maintain) with being useful (something I refer back to).

To be SUCCESSFUL, lists should be easy to add to. Creating a list is a matter of recall: I stream-of-conscious write out all the books I want to read that I remember right now. The easiest possible structure is a single long list of every book I think of.

To be USEFUL, lists need to recommend something. The process goes "Hm I want to read a book" —(check list)—> a particular book to read.

With too many items (~20), going through the list is nontrivial work, so the list is useless. That's when you decide to breakup a list into a superlist of lists, each list with a name.

Grouping Items

There's an extremely important moment when you turn a single long list into a hierarchy of superlists. You have to decide: by what mechanism will I separate books into different groups? This decision is usually done subconsciously. But subconscious defaults are still decisions!

The default subconscious decision is to group by category, by genre. I categorized my books by category. Nonfiction nests either into Math or Psychology, with a "Misc" block to take anything that doesn't fit well into a genre.

Group by category

Category lists are pretty okay at collecting a medium sized number of items (~20 – ~50).

But category lists really aren't that useful, because they fail at recall. Brains don't think in genres. I've never thought "Hm I'd like to read a behavioral psychology book", in the same way I don't think "oh let's listen to some Folktronica" with music.

Brain recall is illegibly associative. Most lookups I want to do are things like "oh I liked the whole NYC suits-doing-business of Power Broker, but I want something less dense". Genre lists are useless at doing that kind of search.

I also quickly run into tricky problems defining categories. I have German language books on my list. Do I put the German-language books in their own German section, and have a German-fiction German-nonfiction subsections, or do I intersperse German language books in the normal lists?

When grouping by category, each book should ideally exist in only one category, and categories should be exactly as specific to make that possible. This is why music genres get more specific by place and era: Polish alt-trance, eg. What if we remove that requirement?

Revealed preferences on Spotify

The best feature of Spotify is the centering of user-created playlists. These are public and like-able, so that popular ones spread.

Spotify creates categorical playlists by genre, like "Grunge Forever" with some Nirvana, Soundgarden, and Alice in Chains. Users, however, overwhelmingly make playlists with names like "wonderfully chaotic", "just sitting in my room" "vaguely experimental: myspace lol 2008 xD" or "norwegian wood" What does it mean for a song to categorically sound like Myspace or a Murakami book? It doesn't mean anything clear, because it's not categorical. These are associative lists.

Associative lists are more useful than category lists, and they're easier to make. They don't need find every song that could fit in the list, only some. Associative lists are an emergent phenomenon, like desire paths.

Group by association

A few months after making the original categorical list and ignoring it, I added a handful of books in a new superlist. It sits at the same super-level as "Fiction" and "Nonfiction".

• Tech Bro canon, the rationalist world
  • Sapiens: A Brief History of Humankind
  • Superintelligence: Paths, Dangers, Strategies
  • Gödel, Escher, Bach: An Eternal Golden Braid
  • Thinking, Fast and Slow by Daniel Kahneman
  • Gang Leader for A Day by Sudhir Venkatesh
  • Andy Weir: the Egg
  • Finite and Infinite Games
  • Zero to One
  • Beginning of Infinity

It's not well grouped by genre. There's mostly nonfiction here, but of different types, and there's even some fiction.

It's more of a gesturing vaguely at something grouping than a legible or describable one. That's what makes these lists useful! They are easier to recall because they more closely match your thoughts, which gesture vaguely at the illegible.

Associative lists are good for collection, because they are easy to create. Each associative list is independent of each other. One list might point at "the melancholy of the big city" and include an Elliott Smith song, another might point to "a cozy high" and include the same Elliott Smith song. You do not have to search through existing lists to create a new one, like with categorical groupings, because the same song can be in multiple groups.

Lists are useful, actually

Lists assist collection and recall. They are especially important tools to deal with excessive amounts of information, which in the Information Age is basically everything. The structure of your lists, therefore, dictates what you know.

People can only do what they remember. Human minds are not default systematic, and the extent to which they are systematic is constrained by what they recall. Associative lists structure your collections more like your thoughts, and in my experience are more useful.

A lot of fields of study, too, are defined by listmaking. What is history if not grouping individual people and events into some useful structure?

Most scholars group history by category: movements and eras and trends. This is the most logically consistent. But what would an associative history look like?

What would history look like if it was less like Wikipedia and more like Spotify playlists?