Surprisingly, LLMs can be used to assess writing for quality. I wouldn't say they're anywhere close to a skilled human, but they can tell the difference between slush pile writing and publishable work—with a granularity of about 15 points on a 100-point scale±—and they can predict commercial success as well as a person can. Traditional publishing takes a strong public stance against AI, but I'm sure they're investigating it for (a) slush triage, (b) comp generation, (c) sales forecasting.
You have to know people in publishing to get a fair read (querying doesn't work) so the replacement of gatekeepers by AI will be an improvement for almost everyone. Of course, they'll never admit that they're doing it, just as college admissions offices still claim they read every application (70+ percent are model rejected.) Still, it's a guarantee. These are underpaid, overworked people and the cognitive load of slush dredging is beyond what anyone can handle.
What's impressive about LLMs isn't that they generate text. That's useful but also not new (remember Markov text bots in the 2000s?) and a bit black-hat, since it basically busts the proof-of-work that language, when it required a human, entailed. Rather, what's impressive is that they read so well—at least, some of the time. The problem is that they're lazy, and also that they're easy to bias. You have to check the work.
LLMs are huge in every field of CS research right now, but they have a flaw that I think people are overlooking: silent failure. You don't get a DivideByZero exception that stops the program; you get articulate bullshit. Even still, the fact that a machine can read forty novels in a minute and have any comprehension—even if it's sometimes lazy, biased comprehension—is extremely impressive.
tag yourself i’m “desolate piss lakes”. the parrot will not critique, it’ll produce a statistically plausible critique.
Surprisingly, LLMs can be used to assess writing for quality. I wouldn't say they're anywhere close to a skilled human, but they can tell the difference between slush pile writing and publishable work—with a granularity of about 15 points on a 100-point scale±—and they can predict commercial success as well as a person can. Traditional publishing takes a strong public stance against AI, but I'm sure they're investigating it for (a) slush triage, (b) comp generation, (c) sales forecasting.
You have to know people in publishing to get a fair read (querying doesn't work) so the replacement of gatekeepers by AI will be an improvement for almost everyone. Of course, they'll never admit that they're doing it, just as college admissions offices still claim they read every application (70+ percent are model rejected.) Still, it's a guarantee. These are underpaid, overworked people and the cognitive load of slush dredging is beyond what anyone can handle.
What's impressive about LLMs isn't that they generate text. That's useful but also not new (remember Markov text bots in the 2000s?) and a bit black-hat, since it basically busts the proof-of-work that language, when it required a human, entailed. Rather, what's impressive is that they read so well—at least, some of the time. The problem is that they're lazy, and also that they're easy to bias. You have to check the work.
LLMs are huge in every field of CS research right now, but they have a flaw that I think people are overlooking: silent failure. You don't get a DivideByZero exception that stops the program; you get articulate bullshit. Even still, the fact that a machine can read forty novels in a minute and have any comprehension—even if it's sometimes lazy, biased comprehension—is extremely impressive.
currently working on some agentic gubbins to explain legacy code so i can appreciate their usefulness in some arenas.
but by critique i mean something that genuinely engages with your ideas - builds your thought model and bends it or extends it or breaks it.