Start to opentracker's memory requirements

* peer lists for v4 and v6 are now lazily allocated and, more importantly, will be freed once there are no more peers
* The downloaded count is now per torrent and not per peer list anymore, moving in what was alignment loss, anyway
* The (now) per torrent downloaded and the per peer list peer and seed count shrunk from size_t to uint32_t
* A helper guards accesses to the peer and seed count to prevent null pointer dereferences
* The base time also shrank from time_t to uint32_t, minutes since the epoch currently fits in 25 bits

1 files changed, 19 insertions, 0 deletions

diff --git a/DESIGN b/DESIGN
index 39d0107..a32eeb8 100644
--- a/DESIGN
+++ b/DESIGN
@@ -84,6 +84,25 @@ finish a sweep in OT_CLEAN_INTERVAL_MINUTES. At the default bucket count that
 is one bucket every 117 ms.
 
 
+A torrent carries its hash, two peer-list pointers, a base and a down count,
+48 bytes on a 64-bit host, and the down count sits in what would otherwise be
+alignment padding. Each family's peer list, 32 bytes plus its peer vector, is
+allocated on that family's first peer and freed when its last peer goes. The
+base and the down count live on the torrent rather than on a list, which is
+what lets a list come and go: a torrent kept only for its download count holds
+no list at all, and a single-family torrent holds one rather than two.
+
+This is paid for in the common shape of a large table. A tracker carrying
+millions of torrents averages close to two peers each, a few crowded swarms
+over a long tail of near-empty ones, so fixed per-torrent cost, not peer
+payload, is most of resident memory. Allocating both lists for every torrent
+and keeping an idle torrent alive for a day to preserve one counter cost, on
+glibc, 168 bytes per torrent before a peer; the layout above costs 48 for an
+empty torrent and 96 for a single-family one. The per-torrent counts are
+uint32: no single torrent holds 2^32 peers, and the four bytes saved against a
+size_t are multiplied by the torrent count. Global totals stay 64-bit.
+
+
 A refused announce is answered over UDP with a well formed announce for an
 empty swarm rather than with the error action. BEP 15 binds a client to the
 interval it is handed and says nothing about what a client should do with an