But today they train with the full github and you get a mix of stuff. If this AI would be so clever then IMO you could have them laarn say Python exactly like a human, a few books and some exercises on python, some books on algorithms, some books on html or whatever tech. Now you as a paying user of this AI tools are not making reviews but probably using them for commercial purposes and it would not be fiar if your proprietary code would use code copy pasted from GPL code. The problem with current AI is that they memorize stuff, there is the case with the AI memorizing an algorithm perfectly, or reciting quotes from Dune and then getting censored. If I read a book and write about it in a blog, it is considered review. >Copyright laws should be amended to allow this scenario.

It's not like Meta can remove these books from the training set without retraining from scratch (or at least the last checkpoint before they were used). It's an exciting development that Meta is releasing a commercial-use version of the model, but I wonder if this is going to cause issues down the road.

IANAL, but this basically sounds like LLaMa was trained on illegally obtained books by Meta's own admission. ThePile, the complaint points out, was described in an EleutherAI paper as being put together from “a copy of the contents of the Bibliotik private tracker.” Bibliotik and the other “shadow libraries” listed, says the lawsuit, are “flagrantly illegal.” The complaint lays out in steps why the plaintiffs believe the datasets have illicit origins - in a Meta paper detailing LLaMA, the company points to sources for its training datasets, one of which is called ThePile, which was assembled by a company called EleutherAI. From the recent story about the Sarah Silverman lawsuit:

Train what shadow of a doubt