• 0 Posts
  • 199 Comments
Joined 1 year ago
cake
Cake day: June 11th, 2023

help-circle








  • We probably shouldn’t let people repair their own brake pads

    What kind of auth-dystopian nonsense is that?

    Repair an insulin pump the wrong way and it will absolutely kill you

    You’re just as dead if you can’t get that insulin pump repaired or replaced because the manufacturer won’t or can’t support it. When they go bankrupt because other customers have sued them into non-existence, you still own the device they manufactured, and you still need it repaired.

    Further, you presume the manufacturer can provide the best repairs. It is entirely possible and plausible that a competing engineer or programmer can improve upon the device, rendering it safer or providing superior operation. Car Mechanics can install a better braking system than the cheap, generic calipers and pads provided by the factory. Repair technicians can replace generic parts of medical devices allowing superior operation.




  • You may need gasoline, but you don’t need BP’s gasoline. By choosing to buy BP’s gasoline, you support everything BP has ever done. Don’t want to support them, buy different gasoline.

    FWIW, I’m not sure if I have a Xitter account or not. I did at one point. Definitely don’t remember a password, and I probably used a former email account that I can no longer access either, so no way of recovering it if it still exists. I have a severe lack of fucks to give about it.

    But, I am pro-pedantry, and your argument kinda sucked.


  • The “collapse” you’re talking about is a reduction in the diversity of the output, which is exactly what we should expect when we impart a bias toward obviously correct answers, and away from obviously incorrect answers.

    Further, that criticism is based on closed-loop feedback, where the LLM is training itself only on it’s own outputs.

    I’m talking about open-loop, where it is also evaluating the responses from the other party.

    Further, the studies whence such criticism comes are based primarily on image generation AIs, not LLMs. Image generation is highly subjective; there is no definitively “right” or “wrong” output, just whether it appeals to the specific observer. An image generator would need to tailor itself to that specific observer.

    LLM sessions deal with far more objective content.

    A functional definition of insanity is doing the same thing over and over and expecting different results. The inability to consider it’s previous interactions denies it the ability to learn from it’s previous behavior. The idea that AIs must not be allowed to train on their own data is functionally insane.


  • Also, with llms there is no “next time” it’s a completely static model.

    It’s only a completely static model if it is not allowed to use it’s own interactions as training data. If it is allowed to use the data acquired from those interactions, it stops being a static model.

    Kids do learn elementary arithmetic by rote memorization. Number theory doesn’t actually develop significantly until somewhere around 3rd to 5th grade, and even then, we don’t place a lot of value on it at that time. We are taught to memorize the multiplication table, for example, because the efficiency of simply knowing that table is far more computationally valuable than the ability to reproduce it at any given time. That rote memorization is mimicry: the child is simply spitting out a previously learned response.

    Remember: LLMs are currently toddlers. They are toddlers with excellent grammar, but they are toddlers.

    Remember also that simple mimicry is an incredibly powerful problem solving method.


  • I can see why you would think that, but to see how it actually goes with a human, look at the interaction between a parent and child, or a teacher and student.

    “Johnny, what’s 2+2?”

    “5?”

    “No, Johnny, try again.”

    “Oh, it’s 4.”

    Turning Johnny into an LLM,nThe next time someone asks, he might not remember 4, but he does remember that “5” consistently gets him a “that’s wrong” response. So does “3”.

    But the only way he knows 5 and 3 gets a negative reaction is by training on his own data, learning from his own mistakes.

    He becomes a better and better mimic, which gets him up to about a 5th grade level of intelligence instead of a toddler.