IPList.cc - Blog

29 Jan 2020 - Updates

After almost a year and millions of daily IP requests it's time for a first blog post. What happened in the past year and what's next. A lot of users also requested information on the database and queries.

• Average API requests: 134 per second

The main architecture consists of one master server constantly parsing and evaluating IP/spam/ASN lists and scanning subnets to update the main database. This main database is distributed (replicated) to multiple other servers. Each backend has a full copy of this information and can answer an API request autonomously. The data is stored in a MariaDB database.

• A single API request requires 10 database queries

The backend shows provides some statistics, it also has a health status for each server including disk and memory usage which are omitted here for brevity.

To achieve acceptable performance all queries use proper indexes and all indexes fit in memory. The database load is usually low, even at times when thousands of users are requesting different IP addresses, the mysqld process (the main MariaDB process) is mostly idle. Although performance is not an issue it's interesting to see how things might be improved. Does caching results or using stored procedures improve performance? As it turns out it does not, at least not yet, this might change when more and more requests are coming in.

• Caching results in Memcached or using a single stored procedure does not result in performance improvement

To keep things simple, each API request opens or reuses one connection to the databases, executes ten queries and returns the result as a JSON object to the user. Sounds good in theory, but what if something goes wrong?

• A growing debug log resulted in a full disk

There was one major incident in the last year. A query debug log was left open by accident. This debug log registers every query performed on the main database. This results in two major problems, it slows down API requests and it quickly fills up the disk. This caused some unresponsive API calls for a few minutes. Fortunately the monitoring system quickly alerted the impending doom of a full disk (albeit a bit too late) and simply disabling the debug log (and removing it!) solved the issue.

• More endpoints to reduce latency

Although the server load is low, the latency can be improved. More servers are being added around the world to send each request to a server near you. This is an ongoing process and will probably be completed soon.