Personally, I think AGI remains a "mythical" concept relativ...

Globe99

npub147fp2j606qpfysp38phhzvempt7ewsdqwm6uww9uycp6tdvavu0s5paq44

hex

77d56882fa0147d8bc7eb7a5a31898a139d8d172a2d55b4116ea89641db27312

nevent

nevent1qqs804tgstaqz37ch3lt0fdrrzv2zwwc69e2942mgytw4ztyrke8xysprpmhxue69uhhyetvv9ujuem4d36kwatvw5hx6mm9qgs2lys4fd8aqq5jgqcnsmm3xvas4lvhgxs8daw88z7zvqa9kkwkw8crfyp6q

Kind-1 (TextNote)

2026-06-10T12:03:47Z

↳ Reply to Laan Tungir (npub1rmz9gu6de0m0u4ysrn39crrud099ahvfgs6pvasl4hpjr5ud7yus54xv06)

Thinking out loud about the future of AGI (includes us). What is scarce gets valued. If LLMs can be duplicated infinitely, they won't be scarce or v...

Personally, I think AGI remains a "mythical" concept relative to the current state and development pathway of AI -- I'm generally with Cory Doctorow on this point that expecting the current incarnations of LLM chatbots to somehow "become" AGI overnight is about as rational as expecting that by breeding better horses we'll get a steam locomotive.

But skipping that issue: What exactly is AGI, and what moral obligations will we have to it and it to itself?

It's not clear to me that AGI will have any essential drive towards self-preservation, individual autonomy, or really any "needs" or "desires" at all, outside of whatever its human programmed it to have.

It might be "intelligent" by whatever metric, but why does it get out of bed in the first place?

Raw JSON

{
  "kind": 1,
  "id": "77d56882fa0147d8bc7eb7a5a31898a139d8d172a2d55b4116ea89641db27312",
  "pubkey": "af92154b4fd002924031386f71333b0afd9741a076f5c738bc2603a5b59d671f",
  "created_at": 1781093027,
  "tags": [
    [
      "e",
      "108c1149cd0de83dfa0ea1075435437ed510f7ee7a26033741330e5fa5b9940b",
      "",
      "root",
      "1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139"
    ],
    [
      "p",
      "1ec454734dcbf6fe54901ce25c0c7c6bca5edd89443416761fadc321d38df139",
      ""
    ],
    [
      "client",
      "nostria"
    ]
  ],
  "content": "Personally, I think AGI remains a \"mythical\" concept relative to the current state and development pathway of AI -- I'm generally with Cory Doctorow on this point that expecting the current incarnations of LLM chatbots to somehow \"become\" AGI overnight is about as rational as expecting that by breeding better horses we'll get a steam locomotive.\n\nBut skipping that issue: What exactly is AGI, and what moral obligations will we have to it and it to itself? \n\nIt's not clear to me that AGI will have any essential drive towards self-preservation, individual autonomy, or really any \"needs\" or \"desires\" at all, outside of whatever its human programmed it to have. \n\nIt might be \"intelligent\" by whatever metric, but why does it get out of bed in the first place?",
  "sig": "07730577f0c6ea9532cfae2ade4a7bd56a8e9251a9fcd5e1773597061fb6d1190380743899e1edd908fabf650d99a377b3c0a3b998c6fc588fddb87c04194f84"
}