Secure Llama Deployment on Debian 12 with Caddy and Custom Domain
Secure Llama Deployment on Debian 12 with Caddy and Custom Domain
Prerequisites
Before we begin, ensure you have the following:
- Debian 12 Server: A fresh installation of Debian 12.
- Domain Name: A registered domain name pointing to your server's IP address.
- Basic Linux Knowledge: Understanding of basic Linux commands and file editing.
- Python: Installed on your Debian server.
Step 1: Initial Server Setup
1.1 Update the System
Start by updating the system to ensure all packages are up-to-date.
1.2 Create a Non-Root User
For security reasons, it's best to create a non-root user.
Switch to the new user:
1.3 Install Necessary Dependencies
Install the essential packages:
Step 2: Clone Ollama for Llama Repository
Navigate to your home directory and clone the repository:
Step 3: Set Up a Python Virtual Environment
Create a virtual environment and activate it:
Step 4: Install Python Dependencies
Install the required Python packages:
Step 5: Configure Environment Variables
Create a .env file in the project root directory and add the necessary environment variables. For example:
Add the following lines, modifying them as necessary:
Step 6: Set Up the Database
6.1 Install PostgreSQL
6.2 Create a Database and User
Switch to the PostgreSQL user and create a database and user:
Run the following SQL commands:
Step 7: Apply Database Migrations
Run the migrations to set up the database schema:
Step 8: Collect Static Files
Collect static files for production:
Step 9: Configure Gunicorn
Install Gunicorn:
Create a Gunicorn service file:
Add the following configuration:
Start and enable the Gunicorn service:
Step 10: Install and Configure Caddy
10.1 Install Caddy
Download and install Caddy:
10.2 Configure Caddy
Create a Caddyfile for your domain:
Add the following configuration:
10.3 Start and Enable Caddy
Start and enable the Caddy service:
Step 11: Monitoring and Maintenance
11.1 Set Up Logging
Ensure you have logging configured for both Gunicorn and Caddy. Gunicorn logs are already configured in the service file. For Caddy, logs are specified in the Caddyfile.
11.2 Monitor Services
Install monitoring tools such as htop and netdata:
11.3 Regular Backups
Set up regular backups for your database and application data. You can use tools like pg_dump for PostgreSQL and rsync for file backups.
Step 12: Troubleshooting
12.1 Gunicorn Not Starting
Check the Gunicorn logs:
Ensure the paths and permissions are correct in the service file.
12.2 Caddy 502 Bad Gateway
Check the Caddy logs:
Ensure the socket file path is correct and that Gunicorn is running.
12.3 Database Connection Issues
Check the PostgreSQL logs:
Ensure the database URL in your .env file is correct.
12.4 SSL Certificate Issues
Check the Caddy logs for certificate renewal issues. Ensure your domain is correctly pointing to your server's IP address.
Conclusion
By following this guide, you have successfully deployed Ollama for Llama on a Debian 12 server with Caddy for SSL termination. Your application is now running in production mode with a secure setup. Regular monitoring and maintenance will ensure your deployment remains stable and secure.
Remember to keep your system updated and review logs regularly to catch any potential issues early. Happy deploying!
