Troubleshooting Paperless-ngx: When Your Documents Get Stuck in the Queue

A beginner's guide to understanding and fixing document processing issues

The Problem: Documents That Never Finish Uploading

If you're reading this, you've probably just set up Paperless-ngx (a fantastic app for organizing your documents digitally) and hit a frustrating snag: when you upload a document, it gets stuck in a queue and never actually processes. The app shows an error message like:

Don't worry—this is actually a common issue, and once you understand what's happening behind the scenes, it's straightforward to fix.

Understanding How Paperless-ngx Works

Before we dive into the solution, let's understand what's supposed to happen when you upload a document. Think of Paperless-ngx like a small factory with different workers, each with a specific job:

The Main Components

1. The Web Server (paperless-ngx.service) This is the front desk of your factory. It's what you see when you open Paperless in your browser. When you upload a document, this component receives it and says, "Okay, I've got a new document. Let me pass this to the workers."

2. The Task Queue (paperless-ngx-task-queue.service) Think of this as the factory floor where the actual work happens. This is powered by something called "Celery," which is a task management system. Workers here do the heavy lifting: OCR (reading text from images), optimizing PDFs, extracting metadata, and organizing everything.

3. The Consumer (paperless-ngx-consumer.service) This component watches a specific folder on your computer. If you drop a file directly into that folder, it notices and says, "Hey, there's a new file here!" and sends it to the task queue.

4. The Scheduler (paperless-ngx-scheduler.service) This handles recurring tasks like checking email for documents, running maintenance jobs, and other scheduled activities.

5. Redis (redis.service) This is the communication hub. Imagine it as a bulletin board where different parts of the system leave messages for each other. The web server posts "New document to process!" and the task queue checks the board to see what work needs to be done.

What Went Wrong?

In this particular case, the problem was that the web server and the task queue couldn't talk to each other properly. Here's what was happening:

1. You uploaded a document through the web interface ✓

2. The web server received it ✓

3. The web server tried to post a message to Redis saying "Process this document" ✗

4. The connection to Redis was broken or misconfigured

5. The document sat in limbo, never making it to the workers

The error message 'NoneType' object has no attribute 'keys' was the web server's way of saying, "I tried to check if the workers are available, but I got no response back—just emptiness (None)."

How We Fixed It

The solution involved several steps to diagnose and repair the communication breakdown:

Step 1: Confirmed Redis Was Running

Redis is crucial for everything to work. We checked:

sudo

Good news: Redis was running! So the problem wasn't that the message board was down—it was that the services weren't connecting to it properly.

Step 2: Found the Services

Paperless-ngx runs multiple services, and we needed to identify all of them:

sudo systemctl list-units | grep

This showed us:

• paperless-ngx-consumer.service

• paperless-ngx-scheduler.service

• paperless-ngx-task-queue.service

• paperless-ngx.service

All were running, but something was still broken.

Step 3: Examined the Logs

Logs are like a diary that programs keep. They tell you what's happening (or what went wrong). We looked at:

sudo tail -200

This revealed two critical things:

1. The task queue WAS working—it was processing scheduled tasks like checking email

2. But no document upload tasks were appearing—meaning documents uploaded through the web weren't reaching the workers

3. A Redis connection error appeared: "Connection reset by peer"

The connection between the web server and the task queue had dropped, likely from a previous crash or misconfiguration.

Step 4: Checked the Configuration

We verified that all the services were configured to use the same Redis database:

sudo cat

The configuration showed:

This means "connect to Redis on this computer, port 6379, using database number 1." All the services needed to use this same address, like making sure everyone in the factory is looking at the same bulletin board.

Step 5: The Fix—Restart Everything in the Right Order

The key insight: when services start up, they establish connections to Redis. If Redis had issues or the connections got stale, simply restarting the services in the proper order would re-establish fresh, working connections.

Here's what we did:

# Stop all Paperless services
sudo systemctl stop paperless-ngx.service
sudo systemctl stop paperless-ngx-task-queue.service
sudo systemctl stop paperless-ngx-consumer.service
sudo systemctl stop paperless-ngx-scheduler.service

# Restart Redis to clear any lingering connection issues
sudo systemctl restart redis

# Start services back up in order, with pauses to let each fully initialize
sudo systemctl start paperless-ngx-task-queue.service
sleep 2
sudo systemctl start paperless-ngx-consumer.service
sleep 2
sudo systemctl start paperless-ngx-scheduler.service
sleep 2
sudo systemctl start

Why this order matters: The task queue needs to be ready before the web server starts, because the web server will immediately try to check if workers are available. Starting them in order ensures each component is ready before the next one tries to communicate with it.

Step 6: Verification

After restarting, we checked the task queue status:

sudo

And we saw active processes running:

• tesseract (OCR software reading text from images)

• gs (Ghostscript, processing PDFs)

• unpaper (cleaning up scanned images)

This meant documents were being processed! We confirmed by looking at the logs:

sudo tail -50

And saw messages like:

Success! Documents were uploading, processing, and being saved.

Key Lessons for Beginners

1. Services Need to Communicate

Modern applications are often made up of multiple services that work together. If they can't communicate (usually through something like Redis or a database), things break in confusing ways.

2. Restart Order Matters

When you have interdependent services, starting them in the right order ensures each is ready before the next tries to use it. Think of it like building with blocks—you need the foundation before you add the walls.

3. Logs Are Your Friend

When something breaks, logs tell you what happened. They might look intimidating at first, but learning to read them is one of the most valuable troubleshooting skills you can develop.

4. Connection Issues Can Be Invisible

The most frustrating problems are when services appear to be running fine individually, but they're not actually talking to each other. This is why we needed to check the connections, not just whether things were running.

Quick Reference: Common Paperless-ngx Commands

Here's a cheat sheet for managing Paperless-ngx on your system:

Check Service Status

sudo systemctl status paperless-ngx.service
sudo systemctl status paperless-ngx-task-queue.service
sudo

View Recent Logs

sudo tail -100 /var/log/paperless-ngx/paperless-ngx.log
sudo tail -100 /var/log/paperless-ngx/paperless-ngx-task-queue.log
sudo tail -100

Restart Everything (The Nuclear Option)

sudo systemctl restart redis
sudo systemctl restart paperless-ngx-task-queue.service
sudo systemctl restart paperless-ngx-consumer.service
sudo systemctl restart paperless-ngx-scheduler.service
sudo systemctl restart

Test Redis Connection

redis-cli ping
# Should return: PONG

Check What's in the Task Queue

redis-cli -n 1 LLEN celery
# Shows how many tasks are waiting

When to Try This Fix

You should try this troubleshooting process if you experience:

• Documents getting stuck in the upload queue

• Error messages about Celery not being available

• "NoneType object has no attribute 'keys'" errors

• Documents that never finish processing

• The system status showing Celery connection issues

Prevention: Keeping Things Running Smoothly

To avoid this issue in the future:

1. Keep your system updated: Run updates regularly through YunoHost's interface

2. Monitor disk space: Full disks can cause Redis and other services to crash

3. Check logs periodically: Catching issues early prevents bigger problems

4. Restart after system updates: Sometimes system updates require service restarts to work properly

Final Thoughts

Troubleshooting technical issues can feel overwhelming when you're new to self-hosting applications. The key is to approach problems systematically:

1. Understand what the system is supposed to do

2. Identify which part isn't working

3. Check the logs to see what it's telling you

4. Try the simplest solutions first (like restarts)

5. Verify the fix worked

In this case, what looked like a complex error with a scary message turned out to be a straightforward communication problem that was solved by restarting services in the right order.

Now your Paperless-ngx installation should be happily processing documents, helping you build your digital paper-free office. Happy organizing!

Have questions or run into different issues? The Paperless-ngx community forum and documentation are excellent resources for additional help.