Statistical AI has gained a lot of interest recently. Many people have spent the past year learning the language and jargon to sound knowledgeable. Some self-proclaimed AI developers – script kiddies – have also started on the MLOps path. However, these newcomers often make the mistake of thinking they are part of an elite and are invulnerable. This may not be the case, as statistical AI may render their efforts obsolete very soon. On the other hand, regular individuals like you and me can benefit from these practical tools in our daily lives. Think of it as having an assistant at your fingertips.

To illustrate my point, this morning, I put my Linux server to work. In less than 20 minutes, I ran a large language model (LLM) at home and tested its performance on my old hardware by asking it some questions. I also evaluated a few models and their quality of responses. I plan to make practical use of it in the scope of my hobbies and will share my experience later.

There are many tools available, and my approach is not exhaustive. It is a simple method that anyone can try now or wait for an even easier method in the future. Imagine a tool that you can download and install on your computer, which can run several models that encode knowledge from their training data. Each model has its pros and cons. Once the tool runs the model of your choice, you can ask it questions as you would with a human interlocutor.

The tool I used is Ollama, which can be downloaded from here (curl -fsSL https://ollama.com/install.sh | sh). I chose Linux because my server runs Linux. You can pick Windows or macOS as well. The time for installation may vary based on your internet speed and system resources. My old workstation turned server has the following specifications: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz (identified using sudo dmidecode | grep CPU), Nvidia GeForce GTX 470 (identified using lspci | grep VGA), and 128 GB of RAM (identified using grep MemTotal /proc/meminfo). It’s not a high-performance machine by any means.

Once installed, you can choose and download a model to run from here. Depending on your resources, it’s advisable to start with smaller models, as larger and more complex models will run slower and may become unusable on your computer. I chose the Google gemma2 model from here and ran it using the command ollama run gemma2. Voila! You now have your own AI expert hosting a large language model for free (well, you still pay the electricity)!

I asked a few questions about programming languages out of personal interest, but you can find the best model for your specific problem and seek help. If you have limited resources at home, you may encounter difficulties running the latest and greatest models. However, you may not always need them. If you do, you can use an xGPT online service. As you can see, it is quite trivial!

Pin