llama.cpp is an open source software library that performs inference on various large language models such as granite, mistral and llama. Cc llama.ccp - Wikipedia