llm-inference

gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.