There’s a brand new paper on Gabriel Abrams’ LLMs, right here is the abstract:
We prompted GPT-4 (a big language mannequin) to play the sport Dictator, a traditional behavioral economics experiment, by taking part in 148 fictional literary characters from the seventeenth to the twenty first century.
Of literary curiosity, this text analyzed the selfishness of characters by century, the relative frequency of persona traits of literary characters and the typical valence of those traits. The article additionally analyzed gender character variations in selfishness.
From an financial/AI perspective, this text generates particular and quantifiable Turing exams that the mannequin handed for zero value impact, lack of malice and altruism, and failed for human sensitivity to relative ordinal place and value elasticity (elasticity is considerably decrease than that of people). Mannequin updates from March to August 2023 had comparatively minor impacts on Turing check outcomes.
There’s a basic and largely monotonic lower in egocentric habits over time amongst literary characters. 50% of seventeenth century characters’ choices are egocentric, in comparison with solely 19% of twenty first century characters’ choices. Total, people confirmed considerably extra egocentric habits than AI characters, with 51% of human choices being egocentric, in comparison with 32% of choices made by AI characters.
Historic literary characters have a surprisingly sturdy internet constructive valence out of the two,785 persona traits generated by GPT-4 (3.2 instances extra constructive than unfavorable). Nevertheless, the valence diverse significantly over the centuries. Essentially the most constructive century, by way of persona traits, was the twenty first – greater than 10 instances the ratio of constructive to unfavorable traits. The least constructive century was the seventeenth with just one.8X. “Empathetic,” “truthful,” and “selfless” have been probably the most overweight traits of the twentieth century. Conversely, “manipulative,” “bold,” and “ruthless” have been probably the most overweight traits within the seventeenth century.
Male characters have been extra egocentric than feminine characters: 35% of male choices have been egocentric, in comparison with solely 24% for feminine characters. The imbalance was highest within the seventeenth century, the place the egocentric choices of women and men have been 62% and 20%, respectively.
This evaluation gives a selected and quantifiable partial Turing check. In some methods, the mannequin appears to be like remarkably like a human being; The primary human traits are the zero value impact, the absence of malice and altruism. Nevertheless, in different methods, GPT-4 displays uncommon or inhumane preferences. The mannequin doesn’t seem to have human sensitivity to relative ordinal place and has considerably lower cost elasticity than people.
Updates to the mannequin in GPT-4 made it barely extra delicate to the ordinal worth, however no more egocentric. The mannequin exhibits consistency of preferences throughout mannequin runs for every character with respect to selfishness.
To which journal may you advise him to ship this text?
