llama cpp

Drag to reposition cover

llama cpp

Llama.cpp is an open-source C/C++ project that lets you run large language models locally on your computer without relying on cloud services. It supports CPU and GPU acceleration, quantized models for faster performance, and works across Windows, macOS, and Linux. Right now, it supports many models (such as LLaMA 2/3, Mistral, etc.) in the GGUF format and can be used via the command line or via Python for chatbots, text generation, and other AI tasks.