U
Unknown
• answered 2 months ago
0
0
(V)RAM requirement for inference: Model size < RAM requirements < Model size * 1.2.
https://huggingface.co/spaces/hf-accelerate/model-memory-usage is helpful
0
0
I would like to find the GPU size required to run an hypothetical LLM, considering all possible factors, like:
I understand how the usual formula I can find around
Space = ((P × 4Bytes) / (32 / Q)) × overhead
does describe some part of the picture, but does not give the full idea down to the details.
Unknown
• answered 2 months ago
0
0
(V)RAM requirement for inference: Model size < RAM requirements < Model size * 1.2.
https://huggingface.co/spaces/hf-accelerate/model-memory-usage is helpful
1