Blog 4: How GenAI Works
Published on:
Learning and machine bias
The case study explains how modern generative AI, or simply gen AI, works “under the hood.” It highlights the strengths, limitations, and biases of the models and dicusses issues of ethics and society.
Discussion Topic: Next-word prediction
Large language models (LLMs) do not reason as presented by popular media. They are probabilistic machines that are trained to predict the most likely next token given local context, not unlike autocomplete. While earlier studies argue for emergent abilities like math or multi-step reasoning, articles like Schaeffer et al (2023) argue that this is a mirage resulting from the choice of research metrics. I don’t quite understand the technical details, but I understand that this is a problem with measurement. This is helpful in understanding LLMs as pattern learners and regurgitators rather than reasonable “thinkers.”
New Discussion Question
If LLMs simply predict the likelihood of words, how does the training data influence their blind spots and biases?
Since models are incapable of comprehending texts, they are limited to their training corpus. A model built on internet texts would generate common language, and from language, ideas. This is also presents the problem of cultural and linguistic bias. Additionally, even if we were to incorporate texts from other languages to enrich the model’s faux “understanding,” there is no clear process of evaluating the translations themselves.
I enjoyed reading the article because it explained the complexicty of AI in an approachable way. I am not confident of my understanding of the research articles, but I found their takeaways helpful for a holistic view of gen AI. This exercise made me further committed to governing LLM use and advocating for critical engagement and bookkeeping.