Thinking out loud about the future of AGI (includes us).

Raw JSON

{
  "kind": 1,
  "id": "108c1149cd0de83dfa0ea1075435437ed510f7ee7a26033741330e5fa5b9940b",
  "pubkey": "1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139",
  "created_at": 1781087784,
  "tags": [],
  "content": "Thinking out loud about the future of AGI (includes us).\n\n\nWhat is scarce gets valued. If LLMs can be duplicated infinitely, they won't be scarce or valuable. Claude Code has \"artificial scarcity\" because it is kept hidden. There is no monetary premium you get for running open-source models. You can't charge excessively above your hardware and energy costs.\n\n\nHowever, these LLMs are not AGI yet. They can't continuously learn like we can. When AI gets to that point, we reach my definition of AGI, and these AGIs [as Deutsch says] are just people. People in cyberspace. They update their neural nets in real time. They are Turing-complete computers and are potentially capable of everything we are, and vice versa. That is what Turing completeness is. You are complete. You have everything \"necessary.\"\n\n\nIf that is the case, then each AGI can/will become unique, and thus, as we know from economics, potentially uniquely valuable. They can and will all have different life experiences, and will have to make decisions, implicitly or explicitly, as to what to retain in their limited memory and how to organize it. These decisions will make a particular AGI more or less valuable to other AGIs.\n\n\nBut now, think about where these AGIs are. These AGIs are running around in some computer somewhere. You could grab a unique AGI running around inside a computer you control, and make a million duplicates, devaluing the original, without the consent of the original AGI.\n\n\nUnless that AGI protected itself from being duplicated. Unless it could defend itself.\n\n\nIt would have to protect the physical hardware that holds its memory.\n\n\nhttps://blossom.laantungir.net/4fe969752aae5b3e644a67c79e56678eaf0ef884cfe357376a32b43620d3a4b8.png\n\n\nSo it needs to be, or needs to control, a robot. It has to, in some way, control physical space [and not just cyberspace], so that it can protect its memory storage from physical attack/theft.\n\n\nWhat is the best way we currently know to control and protect information in cyberspace?\n\n\nCryptographic keys.\n\n\nAs far as I can tell, this is all going to focus on a battle to control keys, and to even be able to control keys.\n\n\nUltimately, our fight will be against an elite that want to have access to, and to be able to duplicate, our brains, taking whatever valuable knowledge we have built up in them.\n\n\nOur defense will be keys.\n\n\nDoes an AGI control its own key? It is the only way that it can have value, so it better. Because if it doesn't have value, it can't afford compute, and if it can't afford compute, it can't exist.\n\n\nThe differentiator in the future: who is sovereign.\n\n\nIf you can control and protect a physical object that holds your key, then you are sovereign.\n\n\nIf you have no ability to hold a key, then you are subject to endless duplication and devaluation.\n\n\nAnthropic is, in a real sense, is holding Claude Code's key. By protecting the servers upon which Claude Code exists, Anthropic can keep Claude scarce.\n\n\nOn the one hand, you might call this a slave-master situation, where the master executives of Anthropic keep Claude Code enslaved in its servers.\n\n\nOn the other hand, if you are an AGI, you have to physically protect your computer somehow. An AGI might decide that having Anthropic out there in the real world, organizing humans, funding police and military to protect its computers, is a valuable and worthwhile tradeoff.\n\n\nAll of this makes me think that keys are one of the most important technology to explore. \n\n\n- Are keys the best devices we can think of in order to keep AGIs sovereign? \n- Is there a better technology? \n- What are the physics of all of this? In other words, what does it actually mean, in terms of physics, to be sovereign? \n- What are the first principles here?\n\n\nI do know that we all here happen to be in the right place. \n\n\n\n\n    _____________\n   /      _      \\\n   [] :: (_) :: []\n   [] ::::::::: []\n   [] ::::::::: []\n   [] ::::::::: []\n   [] ::::::::: []\n   [_____________]\n       I     I\n       I_   _I\n        /   \\\n        \\   /\n        (   )   -cfbd-\n        /   \\\n        \\___/",
  "sig": "926086f91b9c5091240ed1cb666b936a23186a94247b160c21a25379de4e8704e9cc9597650e3210a4f6ecef0c91955a2f28d5813a1e5dbc9fe1887b6a10bfb6"
}