Exploring Open Ollama Servers: A Deep Dive

Last night, we found ourselves diving into the topic of Ollama servers and their security, with a very simple question in mind: could we find any open servers where anyone could just connect and run models off the shelf? What we discovered was both surprising and insightful.

The Hunt Begins

Using Shodan, the search engine for the internet's connected devices, we scanned for open Ollama servers. To our amazement, we identified over 4,000 servers that were accessible without much restriction. Of course, this led us to the next logical question:

What models are they running?

We built a script to automate this process, collecting IP addresses and reaching out to each server using the /api/tags Ollama endpoint to identify the models they were hosting. This process allowed us to gain a snapshot of the types of AI models being openly deployed on these servers.

Model Dominance and Distribution

Our initial analysis revealed some interesting trends in model usage across these open servers.

• Top 10 Most Common Models: Among the thousands of servers, models like Llama emerged as some of the most frequently hosted. These models were clearly popular for their balance between performance and resource demands.

  

• Distribution of Servers by Model Count: We also looked at how many models each server was hosting. A large number of servers were running multiple models, suggesting that many setups are designed for versatility and handling various AI tasks simultaneously.

  

Lots of Servers with Big Models

Interestingly, a good number of these servers were hosting large-scale models, some of which are incredibly resource-intensive. For example, we found servers running models like:

• Llama3.1:405b
• Hermes3:405B
• Deepseek-v2.5:236B-q8_0
• Mistral-large:123B
• Llama3.2-vision:90B and Llama3.2-vision:90B-instruct-fp16

These models, with parameter sizes ranging from 90 billion to over 400 billion, indicate the presence of seriously powerful setups. Running these models requires significant computational power and bandwidth, meaning that many of these open servers are not just hobbyist setups—they are high-capacity, high-stakes systems. Running these types of systems can cost several tens of thousands of dollars per month.

Implications of Open Servers

Finding open servers hosting such advanced models raises some interesting points:

1. Unauthorized Model Pulling: The Ollama API allows anyone to pull models from these open servers, leading to potential misuse of proprietary or advanced models. Unauthorized access to these models is not only unethical but also illegal.

2. Model Deletion and Injection: The API also permits deleting existing models and creating custom ones. This opens the door for malicious actors to delete valuable models or inject harmful ones, compromising the integrity of the server. Such actions can severely disrupt operations and even lead to data corruption or leakage.

3. Infrastructure Exploitation: Running these advanced models requires substantial computational power and bandwidth, which can cost several tens of thousands of dollars per month. Unauthorized use of these resources can lead to significant financial losses for the server owners, effectively hijacking expensive infrastructure for free.

4. Illegal Activity: It is important to note that accessing, modifying, or exploiting these servers without explicit permission is illegal. Leaving these powerful models accessible without restriction can create a significant attack vector, inviting unauthorized use and potential exploitation.

Key Takeaways

Our exploration highlights the dual nature of open AI infrastructure—it can be both a valuable resource and a potential risk. With over 4,000 Ollama servers out there, and many hosting models with billions or even hundreds of billions of parameters, the need for awareness and better security practices becomes apparent.

At Laava, we understand the complexities involved in managing and securing AI projects. We specialize in handling AI infrastructure in a secure and ethical manner, ensuring that advanced models are protected from unauthorized access and misuse. Whether you're building AI solutions or looking to secure your existing setup, we can help you navigate these challenges safely and efficiently.

If you'd like to learn more about how we can help securely build your AI infrastructure, contact us at Laava. We're here to make sure that AI innovation remains secure, ethical, and beneficial for everyone.

At Laava, we believe in pushing the boundaries of AI while respecting security and ethical guidelines. As AI infrastructure becomes increasingly common, securing it will be as important as developing it.

Get Involved

If you’re as fascinated by AI infrastructure as we are, we invite you to join us at Laava. We’re building the next-generation digital workforce, and there’s always more to explore in the evolving world of AI.

Stay tuned for more insights as we continue our journey into the fascinating, sometimes risky world of open AI servers!