Here’s an excellent video from @karpathy@sigmoid.social with some intriguing ways to think about Large Language Models:
- They are a lossy compression of the Internet.
- They could be seen as “… the kernel process of an emerging operating system”. – they coordinate a lot of resources (memory, computational tools) for problem solving. The internet is the “disk”, the context window is “RAM” as working memory.