r/mlscaling May 06 '22

Scale is All You Need OP, D

https://youtu.be/UPlv-lFWITI
17 Upvotes

1

u/[deleted] May 06 '22 edited May 25 '22

[deleted]

2

u/Competitive-Rub-1958 May 06 '22

If a really complicated lookup table ends up reasoning, I doubt think anyone on Earth could care more

-1

u/[deleted] May 07 '22 edited May 25 '22

[deleted]

7

u/Competitive-Rub-1958 May 07 '22

PaLM presents a strong counter-argument, especially seeing its logical inference capabilities. If you have any substantial evidence pointing otherwise, we're all ears

-4

u/[deleted] May 07 '22 edited May 25 '22

[deleted]

8

u/cfoster0 EA May 07 '22

I don't know where the impression that EleutherAI's models are substantially better per-parameter came from. The only cases I've seen good evidence are for tasks where the performance boost seems attributable to the dataset mix.

5

u/Competitive-Rub-1958 May 07 '22

I think he mixed up the Chinchilla findings with EAI models..