summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat
-rw-r--r--.gitignore3
-rw-r--r--DESIGN99
-rw-r--r--README12
-rw-r--r--man4/opentracker.conf.411
-rw-r--r--opentracker.c43
-rw-r--r--opentracker.xcodeproj/project.pbxproj320
-rw-r--r--ot_event.c2
-rw-r--r--ot_fdstate.c7
-rw-r--r--ot_fdstate.h3
-rw-r--r--ot_http.c29
-rw-r--r--ot_http.h7
-rw-r--r--ot_socket.c21
-rw-r--r--ot_socket.h1
-rw-r--r--tests/accept_filter_probe.py110
-rw-r--r--tests/integration/test_expiry.py7
-rw-r--r--tests/t_event.c23
-rw-r--r--tests/t_socket.c43
17 files changed, 387 insertions, 354 deletions
diff --git a/.gitignore b/.gitignore
index b5c40f0..3e469ff 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,5 @@
1*.o
2*.dSYM 1*.dSYM
2*.o
3opentracker 3opentracker
4opentracker.debug 4opentracker.debug
5opentracker.integration 5opentracker.integration
@@ -7,4 +7,5 @@ opentracker.tsan
7proxy 7proxy
8proxy.debug 8proxy.debug
9tests/runner 9tests/runner
10tests/runner.poll
10tests/runner.sanitize 11tests/runner.sanitize
diff --git a/DESIGN b/DESIGN
new file mode 100644
index 0000000..ff37706
--- /dev/null
+++ b/DESIGN
@@ -0,0 +1,99 @@
1opentracker design notes
2========================
3
4Decisions that cost measurement to reach, and the alternatives that were
5tried and rejected.
6
7What the tracker is optimised for
8---------------------------------
9
10UDP carries most of the traffic. An HTTP request costs about ten times a UDP
11announce, 26 microseconds of tracker CPU against 2.5. Nearly all of that is
12the kernel setting up and tearing down a connection rather than tracker
13code.
14
15A connection carries one request and closes. Keep-alive works and is rarely
16asked for.
17
18Statistics are approximate on purpose. Counters are per thread and summed
19when read, so a /stats read can under-report once. Nothing drifts.
20
21
22Decisions
23---------
24
25One UDP socket per worker, where the kernel spreads datagrams over a group
26bound to one address. Cost per datagram then stays flat as workers are added,
270.71 to 0.72 microseconds from one worker to eight. Sharing one socket grows
28it eightfold over the same range, 0.71 to 6.07.
29
30listen.udp.workers defaults to 1. One worker beats none by 19 to 37 per cent
31everywhere, because a blocking read costs one system call where the event
32loop costs two. Beyond one the gain depends on the kernel balancing, so
33raising it is the operator's decision.
34
35A thread parked in a blocking read is woken by a signal, not by closing or
36shutting down its socket:
37
38 mechanism Linux FreeBSD macOS
39 close() blocked blocked 0 ms
40 shutdown(SHUT_RD) 1 ms 1 ms blocked
41 SO_RCVTIMEO works works works
42 pthread_kill 1 ms 1 ms 1 ms
43
44Only the signal works everywhere. A receive timeout also works, but wakes
45every thread on every tick for the life of the process to catch something
46that happens once: at eight workers it doubled idle context switches, 255 to
47495 over thirty seconds.
48
49Each binary is built in one compiler invocation. Without object files a build
50cannot link objects compiled with different FEATURES, which was a real
51failure mode. It costs incremental rebuilds, 0.075 seconds becoming 1.56.
52
53A socket is read or written, never both. http_interest_set replaces an
54add-and-drop pair so that the invariant cannot be broken by accident, and the
55event layer hears about a change once instead of twice.
56
57Accept waits until the client's request has arrived: TCP_DEFER_ACCEPT on
58Linux, the accf_data filter on FreeBSD. Measured against a flood of silent
59connections, the filter serves more requests than no filter, not fewer, so it
60is not the denial of service vector that FreeBSD-SA-02:26 once described.
61
62
63Paths not taken
64---------------
65
66accf_http, the HTTP-aware accept filter. It duplicates a parser the tracker
67has anyway, puts more kernel code in the path, and holds a connection that
68sends "GET /" and stops, which accf_data hands straight over.
69
70recvmmsg and sendmmsg. No measurable gain on either FreeBSD machine, between
71-4 and +10 per cent, and they do not exist on Darwin.
72
73Link time optimisation. It cuts .text by 13 per cent, 93,497 bytes to 81,630,
74and changes throughput not at all. A tracker waits on memory and system
75calls, not on instruction fetch: the announce path is 6.6 KB of code against
76a 32 KB instruction cache at worst.
77
78A directory for object files. bmake has no pattern rules and the Makefile
79builds under FreeBSD's make today. Building each binary in one invocation
80clears the objects out of the tree without giving that up.
81
82A src/ directory. 52 files sharing an ot_ prefix are a module list rather
83than clutter, and build output was most of what made the tree look crowded.
84
85
86Platform differences that cost us code
87--------------------------------------
88
89SO_REUSEPORT means three different things. Linux spreads datagrams over the
90group; FreeBSD permits the bind and needs SO_REUSEPORT_LB to spread. Darwin
91hands every datagram to whichever socket bound last, so a group there starves
92all but one reader.
93
94Waking a blocked reader differs per kernel. see the table above.
95
96Deferring accept is TCP_DEFER_ACCEPT, or accf_data, or nothing.
97
98make is bmake on FreeBSD and GNU make elsewhere, and the Makefile has to
99build under both.
diff --git a/README b/README
index bb8a06e..c84c4e9 100644
--- a/README
+++ b/README
@@ -1,14 +1,8 @@
1This is opentracker. An open bittorrent tracker. 1This is opentracker. An open bittorrent tracker.
2 2
3You need libowfat (http://www.fefe.de/libowfat/).
4
5Steps to go: 3Steps to go:
6 4
7cvs -d :pserver:cvs@cvs.fefe.de:/cvs -z9 co libowfat 5git clone git://erdgeist.org/opentracker
8cd libowfat
9make
10cd ..
11cvs -d:pserver:anoncvs@cvs.erdgeist.org:/home/cvsroot co opentracker
12cd opentracker 6cd opentracker
13make 7make
14./opentracker 8./opentracker
@@ -24,7 +18,3 @@ sysctl kern.ipc.somaxconn=1024
24sysctl kern.ipc.nmbclusters=32768 18sysctl kern.ipc.nmbclusters=32768
25sysctl net.inet.tcp.msl=10000 19sysctl net.inet.tcp.msl=10000
26sysctl kern.maxfiles=10240 20sysctl kern.maxfiles=10240
27
28License information:
29
30Although the libowfat library is under GPL, Felix von Leitner agreed that the compiled binary may be distributed under the same beer ware license as the source code for opentracker. However, we like to hear from happy customers.
diff --git a/man4/opentracker.conf.4 b/man4/opentracker.conf.4
index b28879f..aa82e1d 100644
--- a/man4/opentracker.conf.4
+++ b/man4/opentracker.conf.4
@@ -100,6 +100,17 @@ group only where the answer is yes.
100.Pa tests/udp_scaling.py 100.Pa tests/udp_scaling.py
101in the source distribution sweeps the setting against a running tracker. 101in the source distribution sweeps the setting against a running tracker.
102Measure on the machine that will carry the traffic. 102Measure on the machine that will carry the traffic.
103.Pp
104On FreeBSD the
105.Xr accf_data 9
106filter has the kernel hold a new connection until data arrives on it, so the
107tracker is woken once with the request already there. opentracker uses it
108wherever it is available; enable it with
109.Cm options ACCEPT_FILTER_DATA
110in the kernel or
111.Va kld_list Ns = Ns Qq accf_data
112in
113.Xr rc.conf 5 .
103.Sh EXAMPLES 114.Sh EXAMPLES
104To specify the address opentracker will listen on for both TCP and UDP connections: 115To specify the address opentracker will listen on for both TCP and UDP connections:
105.Dl listen.tcp_udp 0.0.0.0:6969 116.Dl listen.tcp_udp 0.0.0.0:6969
diff --git a/opentracker.c b/opentracker.c
index 27b2570..b47759e 100644
--- a/opentracker.c
+++ b/opentracker.c
@@ -190,29 +190,45 @@ static void handle_write(const int64_t sock) {
190 http_close(sock); 190 http_close(sock);
191} 191}
192 192
193/* The fast path, assuming that the complete request has usually arrived by the time a
194 connection is accepted, so it is answered here outside the event loop. The more load
195 a tracker has, the more likely it is that by the time it accepts, the roundtrip to
196 the peer has completed.
197 If the request could not be completely handled, the stack cookie is promoted to heap
198 and the connection handed over to the event loop to finish */
193static void handle_accept(const int64_t serversocket, struct ot_workstruct *ws) { 199static void handle_accept(const int64_t serversocket, struct ot_workstruct *ws) {
194 struct http_data *cookie; 200 int64_t sock;
195 int64_t sock; 201 ot_ip6 ip;
196 ot_ip6 ip; 202 uint16_t port;
197 uint16_t port;
198 203
199 while ((sock = socket_accept(serversocket, ip, &port)) != -1) { 204 while ((sock = socket_accept(serversocket, ip, &port)) != -1) {
205 struct http_data cookie;
206 uint32_t generation;
200 207
201 if (!(cookie = (struct http_data *)calloc(1, sizeof(struct http_data)))) { 208 memset(&cookie, 0, sizeof(cookie));
202 close((int)sock); 209 memcpy(cookie.ip, ip, sizeof(ot_ip6));
203 continue; 210 cookie.interest = EVENT_IN;
204 } 211 cookie.flag = STRUCT_HTTP_FLAG_ONSTACK;
205 memcpy(cookie->ip, ip, sizeof(ot_ip6));
206 cookie->interest = EVENT_IN;
207 212
208 if (event_add((int)sock, EVENT_IN, fdstate_open(sock, cookie, time_future_seconds(OT_CLIENT_TIMEOUT))) == -1) { 213 generation = fdstate_open(sock, &cookie, time_future_seconds(OT_CLIENT_TIMEOUT));
214 if (!fdstate_claim((int)sock, generation)) {
209 fdstate_close(sock); 215 fdstate_close(sock);
210 free(cookie); 216 event_close((int)sock);
211 close((int)sock);
212 continue; 217 continue;
213 } 218 }
214 219
215 stats_issue_event(ws, EVENT_ACCEPT, FLAG_TCP, (uintptr_t)ip); 220 stats_issue_event(ws, EVENT_ACCEPT, FLAG_TCP, (uintptr_t)ip);
221
222 handle_read(sock, ws);
223
224 /* Gone already if it was answered and closed inline */
225 if (fdstate_generation((int)sock) != generation)
226 continue;
227
228 if (http_adopt(sock, &cookie) == -1)
229 http_close(sock);
230 else
231 fdstate_release((int)sock);
216 } 232 }
217} 233}
218 234
@@ -339,6 +355,7 @@ static void listen_bind_one(ot_listen *entry) {
339 if (entry->proto == FLAG_TCP) { 355 if (entry->proto == FLAG_TCP) {
340 if ((entry->sock = socket_tcp_listen(entry->ip, entry->port, sharing)) == -1) 356 if ((entry->sock = socket_tcp_listen(entry->ip, entry->port, sharing)) == -1)
341 panic("socket_tcp_listen"); 357 panic("socket_tcp_listen");
358 socket_defer_accept((int)entry->sock);
342 } else { 359 } else {
343 if ((entry->sock = socket_udp_bind(entry->ip, entry->port, sharing)) == -1) 360 if ((entry->sock = socket_udp_bind(entry->ip, entry->port, sharing)) == -1)
344 panic("socket_udp_bind"); 361 panic("socket_udp_bind");
diff --git a/opentracker.xcodeproj/project.pbxproj b/opentracker.xcodeproj/project.pbxproj
deleted file mode 100644
index 3706a3f..0000000
--- a/opentracker.xcodeproj/project.pbxproj
+++ /dev/null
@@ -1,320 +0,0 @@
1// !$*UTF8*$!
2{
3 archiveVersion = 1;
4 classes = {
5 };
6 objectVersion = 44;
7 objects = {
8
9/* Begin PBXBuildFile section */
10 6520B7530D036AAF00A43B1F /* libowfat.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 6520B7520D036AAF00A43B1F /* libowfat.a */; };
11 653A320C0CE7F475007F0D03 /* ot_accesslist.c in Sources */ = {isa = PBXBuildFile; fileRef = 653A320B0CE7F475007F0D03 /* ot_accesslist.c */; };
12 653A56B50CE28EC5000CF140 /* ot_iovec.c in Sources */ = {isa = PBXBuildFile; fileRef = 653A56B40CE28EC5000CF140 /* ot_iovec.c */; };
13 654A80890CD832FD009035DE /* opentracker.c in Sources */ = {isa = PBXBuildFile; fileRef = 654A80840CD832FC009035DE /* opentracker.c */; };
14 654A808A0CD832FD009035DE /* scan_urlencoded_query.c in Sources */ = {isa = PBXBuildFile; fileRef = 654A80850CD832FC009035DE /* scan_urlencoded_query.c */; };
15 654A808B0CD832FD009035DE /* trackerlogic.c in Sources */ = {isa = PBXBuildFile; fileRef = 654A80870CD832FC009035DE /* trackerlogic.c */; };
16 65542D8B0CE078E800469330 /* ot_vector.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542D8A0CE078E800469330 /* ot_vector.c */; };
17 65542D930CE07CED00469330 /* ot_mutex.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542D8F0CE07CED00469330 /* ot_mutex.c */; };
18 65542D940CE07CED00469330 /* ot_stats.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542D910CE07CED00469330 /* ot_stats.c */; };
19 65542E750CE08B9100469330 /* ot_clean.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542E740CE08B9100469330 /* ot_clean.c */; };
20 65542EE80CE0CA6B00469330 /* ot_udp.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542EE70CE0CA6B00469330 /* ot_udp.c */; };
21 65542F920CE17CA900469330 /* ot_fullscrape.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542F910CE17CA900469330 /* ot_fullscrape.c */; };
22 65B8DF3C0D0310D20017149E /* ot_http.c in Sources */ = {isa = PBXBuildFile; fileRef = 65B8DF3B0D0310D20017149E /* ot_http.c */; };
23 65FA33990E7EF09200F7D5A5 /* ot_livesync.c in Sources */ = {isa = PBXBuildFile; fileRef = 65FA33980E7EF09200F7D5A5 /* ot_livesync.c */; };
24 8DD76FB00486AB0100D96B5E /* opentracker.1 in CopyFiles */ = {isa = PBXBuildFile; fileRef = C6A0FF2C0290799A04C91782 /* opentracker.1 */; };
25/* End PBXBuildFile section */
26
27/* Begin PBXCopyFilesBuildPhase section */
28 8DD76FAF0486AB0100D96B5E /* CopyFiles */ = {
29 isa = PBXCopyFilesBuildPhase;
30 buildActionMask = 8;
31 dstPath = /usr/share/man/man1/;
32 dstSubfolderSpec = 0;
33 files = (
34 8DD76FB00486AB0100D96B5E /* opentracker.1 in CopyFiles */,
35 );
36 runOnlyForDeploymentPostprocessing = 1;
37 };
38/* End PBXCopyFilesBuildPhase section */
39
40/* Begin PBXFileReference section */
41 6520B7520D036AAF00A43B1F /* libowfat.a */ = {isa = PBXFileReference; lastKnownFileType = archive.ar; name = libowfat.a; path = ../libowfat/libowfat.a; sourceTree = SOURCE_ROOT; };
42 653A320A0CE7F475007F0D03 /* ot_accesslist.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_accesslist.h; sourceTree = "<group>"; };
43 653A320B0CE7F475007F0D03 /* ot_accesslist.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_accesslist.c; sourceTree = "<group>"; };
44 653A56AC0CE201FF000CF140 /* opentracker */ = {isa = PBXFileReference; explicitFileType = "compiled.mach-o.executable"; includeInIndex = 0; path = opentracker; sourceTree = BUILT_PRODUCTS_DIR; };
45 653A56B30CE28EC5000CF140 /* ot_iovec.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_iovec.h; sourceTree = "<group>"; };
46 653A56B40CE28EC5000CF140 /* ot_iovec.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_iovec.c; sourceTree = "<group>"; };
47 654A80840CD832FC009035DE /* opentracker.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = opentracker.c; sourceTree = "<group>"; };
48 654A80850CD832FC009035DE /* scan_urlencoded_query.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = scan_urlencoded_query.c; sourceTree = "<group>"; };
49 654A80860CD832FC009035DE /* scan_urlencoded_query.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = scan_urlencoded_query.h; sourceTree = "<group>"; };
50 654A80870CD832FC009035DE /* trackerlogic.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = trackerlogic.c; sourceTree = "<group>"; };
51 654A80880CD832FC009035DE /* trackerlogic.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = trackerlogic.h; sourceTree = "<group>"; };
52 65542D890CE078E800469330 /* ot_vector.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_vector.h; sourceTree = "<group>"; };
53 65542D8A0CE078E800469330 /* ot_vector.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_vector.c; sourceTree = "<group>"; };
54 65542D8F0CE07CED00469330 /* ot_mutex.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_mutex.c; sourceTree = "<group>"; };
55 65542D900CE07CED00469330 /* ot_mutex.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_mutex.h; sourceTree = "<group>"; };
56 65542D910CE07CED00469330 /* ot_stats.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_stats.c; sourceTree = "<group>"; };
57 65542D920CE07CED00469330 /* ot_stats.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_stats.h; sourceTree = "<group>"; };
58 65542E730CE08B9100469330 /* ot_clean.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_clean.h; sourceTree = "<group>"; };
59 65542E740CE08B9100469330 /* ot_clean.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_clean.c; sourceTree = "<group>"; };
60 65542EE60CE0CA6B00469330 /* ot_udp.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_udp.h; sourceTree = "<group>"; };
61 65542EE70CE0CA6B00469330 /* ot_udp.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_udp.c; sourceTree = "<group>"; };
62 65542F900CE17CA900469330 /* ot_fullscrape.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_fullscrape.h; sourceTree = "<group>"; };
63 65542F910CE17CA900469330 /* ot_fullscrape.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_fullscrape.c; sourceTree = "<group>"; };
64 65B8DF3A0D0310D20017149E /* ot_http.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_http.h; sourceTree = "<group>"; };
65 65B8DF3B0D0310D20017149E /* ot_http.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_http.c; sourceTree = "<group>"; };
66 65FA33970E7EF09200F7D5A5 /* ot_livesync.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_livesync.h; sourceTree = "<group>"; };
67 65FA33980E7EF09200F7D5A5 /* ot_livesync.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_livesync.c; sourceTree = "<group>"; };
68 C6A0FF2C0290799A04C91782 /* opentracker.1 */ = {isa = PBXFileReference; lastKnownFileType = text.man; path = opentracker.1; sourceTree = "<group>"; };
69/* End PBXFileReference section */
70
71/* Begin PBXFrameworksBuildPhase section */
72 8DD76FAD0486AB0100D96B5E /* Frameworks */ = {
73 isa = PBXFrameworksBuildPhase;
74 buildActionMask = 2147483647;
75 files = (
76 6520B7530D036AAF00A43B1F /* libowfat.a in Frameworks */,
77 );
78 runOnlyForDeploymentPostprocessing = 0;
79 };
80/* End PBXFrameworksBuildPhase section */
81
82/* Begin PBXGroup section */
83 08FB7794FE84155DC02AAC07 /* opentracker */ = {
84 isa = PBXGroup;
85 children = (
86 65542D810CE0786F00469330 /* Headers */,
87 08FB7795FE84155DC02AAC07 /* Source */,
88 92762AC9104EDED700FDCB60 /* Libraries */,
89 C6A0FF2B0290797F04C91782 /* Documentation */,
90 653A56AD0CE201FF000CF140 /* Products */,
91 );
92 name = opentracker;
93 sourceTree = "<group>";
94 };
95 08FB7795FE84155DC02AAC07 /* Source */ = {
96 isa = PBXGroup;
97 children = (
98 654A80840CD832FC009035DE /* opentracker.c */,
99 653A320B0CE7F475007F0D03 /* ot_accesslist.c */,
100 65542E740CE08B9100469330 /* ot_clean.c */,
101 65542F910CE17CA900469330 /* ot_fullscrape.c */,
102 65B8DF3B0D0310D20017149E /* ot_http.c */,
103 65FA33980E7EF09200F7D5A5 /* ot_livesync.c */,
104 653A56B40CE28EC5000CF140 /* ot_iovec.c */,
105 65542D8F0CE07CED00469330 /* ot_mutex.c */,
106 65542D910CE07CED00469330 /* ot_stats.c */,
107 65542EE70CE0CA6B00469330 /* ot_udp.c */,
108 65542D8A0CE078E800469330 /* ot_vector.c */,
109 654A80850CD832FC009035DE /* scan_urlencoded_query.c */,
110 654A80870CD832FC009035DE /* trackerlogic.c */,
111 );
112 name = Source;
113 sourceTree = "<group>";
114 };
115 653A56AD0CE201FF000CF140 /* Products */ = {
116 isa = PBXGroup;
117 children = (
118 653A56AC0CE201FF000CF140 /* opentracker */,
119 );
120 name = Products;
121 sourceTree = "<group>";
122 };
123 65542D810CE0786F00469330 /* Headers */ = {
124 isa = PBXGroup;
125 children = (
126 653A320A0CE7F475007F0D03 /* ot_accesslist.h */,
127 65542E730CE08B9100469330 /* ot_clean.h */,
128 65542F900CE17CA900469330 /* ot_fullscrape.h */,
129 65B8DF3A0D0310D20017149E /* ot_http.h */,
130 653A56B30CE28EC5000CF140 /* ot_iovec.h */,
131 65FA33970E7EF09200F7D5A5 /* ot_livesync.h */,
132 65542D900CE07CED00469330 /* ot_mutex.h */,
133 65542D920CE07CED00469330 /* ot_stats.h */,
134 65542EE60CE0CA6B00469330 /* ot_udp.h */,
135 65542D890CE078E800469330 /* ot_vector.h */,
136 654A80860CD832FC009035DE /* scan_urlencoded_query.h */,
137 654A80880CD832FC009035DE /* trackerlogic.h */,
138 );
139 name = Headers;
140 sourceTree = "<group>";
141 };
142 92762AC9104EDED700FDCB60 /* Libraries */ = {
143 isa = PBXGroup;
144 children = (
145 6520B7520D036AAF00A43B1F /* libowfat.a */,
146 );
147 name = Libraries;
148 sourceTree = "<group>";
149 };
150 C6A0FF2B0290797F04C91782 /* Documentation */ = {
151 isa = PBXGroup;
152 children = (
153 C6A0FF2C0290799A04C91782 /* opentracker.1 */,
154 );
155 name = Documentation;
156 sourceTree = "<group>";
157 };
158/* End PBXGroup section */
159
160/* Begin PBXNativeTarget section */
161 8DD76FA90486AB0100D96B5E /* opentracker */ = {
162 isa = PBXNativeTarget;
163 buildConfigurationList = 1DEB928508733DD80010E9CD /* Build configuration list for PBXNativeTarget "opentracker" */;
164 buildPhases = (
165 8DD76FAB0486AB0100D96B5E /* Sources */,
166 8DD76FAD0486AB0100D96B5E /* Frameworks */,
167 8DD76FAF0486AB0100D96B5E /* CopyFiles */,
168 );
169 buildRules = (
170 );
171 dependencies = (
172 );
173 name = opentracker;
174 productInstallPath = "$(HOME)/bin";
175 productName = opentracker;
176 productReference = 653A56AC0CE201FF000CF140 /* opentracker */;
177 productType = "com.apple.product-type.tool";
178 };
179/* End PBXNativeTarget section */
180
181/* Begin PBXProject section */
182 08FB7793FE84155DC02AAC07 /* Project object */ = {
183 isa = PBXProject;
184 buildConfigurationList = 1DEB928908733DD80010E9CD /* Build configuration list for PBXProject "opentracker" */;
185 compatibilityVersion = "Xcode 3.0";
186 hasScannedForEncodings = 1;
187 mainGroup = 08FB7794FE84155DC02AAC07 /* opentracker */;
188 productRefGroup = 653A56AD0CE201FF000CF140 /* Products */;
189 projectDirPath = "";
190 projectRoot = "";
191 targets = (
192 8DD76FA90486AB0100D96B5E /* opentracker */,
193 );
194 };
195/* End PBXProject section */
196
197/* Begin PBXSourcesBuildPhase section */
198 8DD76FAB0486AB0100D96B5E /* Sources */ = {
199 isa = PBXSourcesBuildPhase;
200 buildActionMask = 2147483647;
201 files = (
202 654A80890CD832FD009035DE /* opentracker.c in Sources */,
203 654A808A0CD832FD009035DE /* scan_urlencoded_query.c in Sources */,
204 654A808B0CD832FD009035DE /* trackerlogic.c in Sources */,
205 65542D8B0CE078E800469330 /* ot_vector.c in Sources */,
206 65542D930CE07CED00469330 /* ot_mutex.c in Sources */,
207 65542D940CE07CED00469330 /* ot_stats.c in Sources */,
208 65542E750CE08B9100469330 /* ot_clean.c in Sources */,
209 65542EE80CE0CA6B00469330 /* ot_udp.c in Sources */,
210 65542F920CE17CA900469330 /* ot_fullscrape.c in Sources */,
211 653A56B50CE28EC5000CF140 /* ot_iovec.c in Sources */,
212 653A320C0CE7F475007F0D03 /* ot_accesslist.c in Sources */,
213 65B8DF3C0D0310D20017149E /* ot_http.c in Sources */,
214 65FA33990E7EF09200F7D5A5 /* ot_livesync.c in Sources */,
215 );
216 runOnlyForDeploymentPostprocessing = 0;
217 };
218/* End PBXSourcesBuildPhase section */
219
220/* Begin XCBuildConfiguration section */
221 1DEB928608733DD80010E9CD /* Debug */ = {
222 isa = XCBuildConfiguration;
223 buildSettings = {
224 COPY_PHASE_STRIP = NO;
225 GCC_DYNAMIC_NO_PIC = NO;
226 GCC_ENABLE_FIX_AND_CONTINUE = YES;
227 GCC_MODEL_TUNING = G5;
228 GCC_OPTIMIZATION_LEVEL = 0;
229 INSTALL_PATH = /usr/local/bin;
230 LIBRARY_SEARCH_PATHS = (
231 "$(inherited)",
232 "\"$(SRCROOT)/../libowfat\"",
233 );
234 PRODUCT_NAME = opentracker;
235 ZERO_LINK = YES;
236 };
237 name = Debug;
238 };
239 1DEB928708733DD80010E9CD /* Release */ = {
240 isa = XCBuildConfiguration;
241 buildSettings = {
242 DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
243 GCC_ENABLE_FIX_AND_CONTINUE = YES;
244 GCC_MODEL_TUNING = G5;
245 INSTALL_PATH = /usr/local/bin;
246 LIBRARY_SEARCH_PATHS = (
247 "$(inherited)",
248 "\"$(SRCROOT)/../libowfat\"",
249 );
250 PRODUCT_NAME = opentracker;
251 };
252 name = Release;
253 };
254 1DEB928A08733DD80010E9CD /* Debug */ = {
255 isa = XCBuildConfiguration;
256 buildSettings = {
257 ARCHS = "$(ONLY_ACTIVE_ARCH_PRE_XCODE_3_1)";
258 GCC_PREPROCESSOR_DEFINITIONS = (
259 WANT_ACCESSLIST_WHITE,
260 WANT_IP_FROM_QUERY_STRING,
261 WANT_FULLSCRAPE,
262 );
263 GCC_VERSION = com.apple.compilers.llvm.clang.1_0;
264 GCC_WARN_ABOUT_RETURN_TYPE = YES;
265 GCC_WARN_UNUSED_VARIABLE = YES;
266 HEADER_SEARCH_PATHS = ../libowfat/;
267 LIBRARY_SEARCH_PATHS = ../libowfat/;
268 ONLY_ACTIVE_ARCH_PRE_XCODE_3_1 = "$(NATIVE_ARCH_ACTUAL)";
269 PREBINDING = NO;
270 SDKROOT = "$(DEVELOPER_SDK_DIR)/MacOSX10.6.sdk";
271 };
272 name = Debug;
273 };
274 1DEB928B08733DD80010E9CD /* Release */ = {
275 isa = XCBuildConfiguration;
276 buildSettings = {
277 ARCHS = "$(ONLY_ACTIVE_ARCH_PRE_XCODE_3_1)";
278 DEAD_CODE_STRIPPING = NO;
279 GCC_PREPROCESSOR_DEFINITIONS = (
280 WANT_ACCESSLIST_WHITE,
281 WANT_IP_FROM_QUERY_STRING,
282 WANT_FULLSCRAPE,
283 );
284 GCC_VERSION = com.apple.compilers.llvm.clang.1_0;
285 GCC_WARN_ABOUT_RETURN_TYPE = YES;
286 GCC_WARN_UNUSED_VARIABLE = YES;
287 HEADER_SEARCH_PATHS = ../libowfat/;
288 LIBRARY_SEARCH_PATHS = ../libowfat/;
289 ONLY_ACTIVE_ARCH_PRE_XCODE_3_1 = "$(NATIVE_ARCH_ACTUAL)";
290 PREBINDING = NO;
291 SDKROOT = "$(DEVELOPER_SDK_DIR)/MacOSX10.6.sdk";
292 ZERO_LINK = NO;
293 };
294 name = Release;
295 };
296/* End XCBuildConfiguration section */
297
298/* Begin XCConfigurationList section */
299 1DEB928508733DD80010E9CD /* Build configuration list for PBXNativeTarget "opentracker" */ = {
300 isa = XCConfigurationList;
301 buildConfigurations = (
302 1DEB928608733DD80010E9CD /* Debug */,
303 1DEB928708733DD80010E9CD /* Release */,
304 );
305 defaultConfigurationIsVisible = 0;
306 defaultConfigurationName = Release;
307 };
308 1DEB928908733DD80010E9CD /* Build configuration list for PBXProject "opentracker" */ = {
309 isa = XCConfigurationList;
310 buildConfigurations = (
311 1DEB928A08733DD80010E9CD /* Debug */,
312 1DEB928B08733DD80010E9CD /* Release */,
313 );
314 defaultConfigurationIsVisible = 0;
315 defaultConfigurationName = Release;
316 };
317/* End XCConfigurationList section */
318 };
319 rootObject = 08FB7793FE84155DC02AAC07 /* Project object */;
320}
diff --git a/ot_event.c b/ot_event.c
index 8f55817..b4c4a55 100644
--- a/ot_event.c
+++ b/ot_event.c
@@ -171,7 +171,7 @@ static int event_change(int fd, int what, uint32_t generation, int deleting) {
171 return 0; 171 return 0;
172} 172}
173 173
174int event_add(int fd, int what, uint32_t generation) { return event_change(fd, what, generation, 0); } 174int event_add(int fd, int what, uint32_t generation) { return what ? event_change(fd, what, generation, 0) : 0; }
175int event_modify(int fd, int what, uint32_t generation) { return event_change(fd, what, generation, 0); } 175int event_modify(int fd, int what, uint32_t generation) { return event_change(fd, what, generation, 0); }
176int event_del(int fd) { return event_change(fd, 0, 0, 1); } 176int event_del(int fd) { return event_change(fd, 0, 0, 1); }
177 177
diff --git a/ot_fdstate.c b/ot_fdstate.c
index 537bcb0..7d742da 100644
--- a/ot_fdstate.c
+++ b/ot_fdstate.c
@@ -186,6 +186,13 @@ void *fdstate_cookie(int fd) {
186 return slot ? atomic_load_explicit(&slot->cookie, memory_order_relaxed) : NULL; 186 return slot ? atomic_load_explicit(&slot->cookie, memory_order_relaxed) : NULL;
187} 187}
188 188
189void fdstate_set_cookie(int fd, void *cookie) {
190 ot_fdslot *slot = fdstate_slot(fd);
191
192 if (slot)
193 atomic_store_explicit(&slot->cookie, cookie, memory_order_relaxed);
194}
195
189void fdstate_set_expiry(int fd, ot_time expiry) { 196void fdstate_set_expiry(int fd, ot_time expiry) {
190 ot_fdslot *slot = fdstate_slot(fd); 197 ot_fdslot *slot = fdstate_slot(fd);
191 198
diff --git a/ot_fdstate.h b/ot_fdstate.h
index c09f018..93890d1 100644
--- a/ot_fdstate.h
+++ b/ot_fdstate.h
@@ -56,6 +56,9 @@ int fdstate_next_expired(ot_time now, int *cursor);
56 56
57/* Both only valid while the caller holds the descriptor. */ 57/* Both only valid while the caller holds the descriptor. */
58void *fdstate_cookie(int fd); 58void *fdstate_cookie(int fd);
59
60/* Hand the slot a different cookie, for a connection whose state has moved */
61void fdstate_set_cookie(int fd, void *cookie);
59void fdstate_set_expiry(int fd, ot_time expiry); 62void fdstate_set_expiry(int fd, ot_time expiry);
60 63
61#endif 64#endif
diff --git a/ot_http.c b/ot_http.c
index 66f0102..bb6ff24 100644
--- a/ot_http.c
+++ b/ot_http.c
@@ -39,9 +39,25 @@ ssize_t g_stats_path_len;
39 39
40enum { SUCCESS_HTTP_HEADER_LENGTH = 80, SUCCESS_HTTP_SIZE_OFF = 17 }; 40enum { SUCCESS_HTTP_HEADER_LENGTH = 80, SUCCESS_HTTP_SIZE_OFF = 17 };
41 41
42/* Give up a connection: the one place that knows everything hanging off it. 42int http_adopt(const int64_t sock, struct http_data *cookie) {
43 The slot is released before anything is freed, so that a late event or the 43 struct http_data *heap = malloc(sizeof(*heap));
44 expiry sweep finds a closed descriptor rather than a freed cookie. */ 44
45 if (!heap)
46 return -1;
47
48 *heap = *cookie;
49 heap->flag &= ~STRUCT_HTTP_FLAG_ONSTACK;
50 fdstate_set_cookie((int)sock, heap);
51
52 if (event_add((int)sock, heap->interest, fdstate_generation((int)sock)) == -1) {
53 fdstate_set_cookie((int)sock, cookie);
54 free(heap);
55 return -1;
56 }
57 return 0;
58}
59
60/* Give up a connection: the one place that knows everything hanging off it. */
45void http_close(const int64_t sock) { 61void http_close(const int64_t sock) {
46 struct http_data *cookie = fdstate_cookie(sock); 62 struct http_data *cookie = fdstate_cookie(sock);
47 63
@@ -54,7 +70,8 @@ void http_close(const int64_t sock) {
54 buf_reset(&cookie->request); 70 buf_reset(&cookie->request);
55 if (cookie->flag & (STRUCT_HTTP_FLAG_WAITINGFORTASK | STRUCT_HTTP_FLAG_CHUNKED_IN_TRANSFER)) 71 if (cookie->flag & (STRUCT_HTTP_FLAG_WAITINGFORTASK | STRUCT_HTTP_FLAG_CHUNKED_IN_TRANSFER))
56 mutex_workqueue_canceltask(sock); 72 mutex_workqueue_canceltask(sock);
57 free(cookie); 73 if (!(cookie->flag & STRUCT_HTTP_FLAG_ONSTACK))
74 free(cookie);
58 } 75 }
59 76
60 event_close((int)sock); 77 event_close((int)sock);
@@ -140,6 +157,10 @@ void http_interest_set(const int64_t sock, struct http_data *cookie, int what) {
140 if (cookie->interest == what) 157 if (cookie->interest == what)
141 return; 158 return;
142 cookie->interest = what; 159 cookie->interest = what;
160 /* Still on a stack, so not in the event loop yet: whoever adopts this
161 connection registers whatever it asked for last */
162 if (cookie->flag & STRUCT_HTTP_FLAG_ONSTACK)
163 return;
143 event_modify((int)sock, what, fdstate_generation((int)sock)); 164 event_modify((int)sock, what, fdstate_generation((int)sock));
144} 165}
145 166
diff --git a/ot_http.h b/ot_http.h
index ba16de4..82d645b 100644
--- a/ot_http.h
+++ b/ot_http.h
@@ -22,7 +22,8 @@ typedef enum {
22 STRUCT_HTTP_FLAG_ZSTD = 8, 22 STRUCT_HTTP_FLAG_ZSTD = 8,
23 STRUCT_HTTP_FLAG_CHUNKED = 16, 23 STRUCT_HTTP_FLAG_CHUNKED = 16,
24 STRUCT_HTTP_FLAG_CHUNKED_IN_TRANSFER = 32, 24 STRUCT_HTTP_FLAG_CHUNKED_IN_TRANSFER = 32,
25 STRUCT_HTTP_FLAG_KEEPALIVE = 64 25 STRUCT_HTTP_FLAG_KEEPALIVE = 64,
26 STRUCT_HTTP_FLAG_ONSTACK = 128
26} STRUCT_HTTP_FLAG; 27} STRUCT_HTTP_FLAG;
27 28
28struct http_data { 29struct http_data {
@@ -39,6 +40,10 @@ void http_interest_set(const int64_t s, struct http_data *cookie, int
39/* Close a connection and release everything that belongs to it */ 40/* Close a connection and release everything that belongs to it */
40void http_close(const int64_t s); 41void http_close(const int64_t s);
41 42
43/* Promote a stack cookie to a heap one for persistence and handing it over
44 to the event loop. */
45int http_adopt(const int64_t s, struct http_data *cookie);
46
42ssize_t http_handle_request(const int64_t s, struct ot_workstruct *ws); 47ssize_t http_handle_request(const int64_t s, struct ot_workstruct *ws);
43ssize_t http_sendiovecdata(const int64_t s, struct ot_workstruct *ws, size_t iovec_entries, struct iovec *iovector, int is_partial); 48ssize_t http_sendiovecdata(const int64_t s, struct ot_workstruct *ws, size_t iovec_entries, struct iovec *iovector, int is_partial);
44ssize_t http_issue_error(const int64_t s, struct ot_workstruct *ws, int code); 49ssize_t http_issue_error(const int64_t s, struct ot_workstruct *ws, int code);
diff --git a/ot_socket.c b/ot_socket.c
index 47ed26d..48ffc27 100644
--- a/ot_socket.c
+++ b/ot_socket.c
@@ -8,6 +8,7 @@
8#include <errno.h> 8#include <errno.h>
9#include <fcntl.h> 9#include <fcntl.h>
10#include <netinet/in.h> 10#include <netinet/in.h>
11#include <netinet/tcp.h>
11#include <string.h> 12#include <string.h>
12#include <sys/socket.h> 13#include <sys/socket.h>
13#include <unistd.h> 14#include <unistd.h>
@@ -120,6 +121,26 @@ static int socket_bind_v6(int s, const ot_ip6 ip, uint16_t port, socket_sharing
120 return bind(s, (struct sockaddr *)&sa, sizeof(sa)); 121 return bind(s, (struct sockaddr *)&sa, sizeof(sa));
121} 122}
122 123
124/* Defer the listening socket reporting it to be ready to accept until enough
125 sensible data is available for an accept+read pair. Where available. */
126int socket_defer_accept(int s) {
127#if defined(TCP_DEFER_ACCEPT)
128 int patience = OT_CLIENT_TIMEOUT;
129
130 return setsockopt(s, IPPROTO_TCP, TCP_DEFER_ACCEPT, &patience, sizeof(patience));
131#elif defined(SO_ACCEPTFILTER)
132 struct accept_filter_arg filter;
133 const char name[sizeof(filter.af_name)] = "dataready";
134
135 memset(&filter, 0, sizeof(filter));
136 memcpy(filter.af_name, name, sizeof(name));
137 return setsockopt(s, SOL_SOCKET, SO_ACCEPTFILTER, &filter, sizeof(filter));
138#else
139 (void)s;
140 return -1;
141#endif
142}
143
123int socket_tcp_listen(const ot_ip6 ip, uint16_t port, socket_sharing sharing) { 144int socket_tcp_listen(const ot_ip6 ip, uint16_t port, socket_sharing sharing) {
124 int s = socket_v6(SOCK_STREAM); 145 int s = socket_v6(SOCK_STREAM);
125 146
diff --git a/ot_socket.h b/ot_socket.h
index f73a5c9..5bbffe1 100644
--- a/ot_socket.h
+++ b/ot_socket.h
@@ -38,6 +38,7 @@ int socket_tcp_connect(const ot_ip6 srcip, uint16_t srcport, const ot_ip6 ds
38int socket_is_connected(int s); 38int socket_is_connected(int s);
39int socket_accept(int s, ot_ip6 ip, uint16_t *port); 39int socket_accept(int s, ot_ip6 ip, uint16_t *port);
40int socket_set_blocking(int s, int blocking); 40int socket_set_blocking(int s, int blocking);
41int socket_defer_accept(int s);
41 42
42ssize_t socket_recv(int s, void *buf, size_t len, ot_ip6 ip, uint16_t *port); 43ssize_t socket_recv(int s, void *buf, size_t len, ot_ip6 ip, uint16_t *port);
43ssize_t socket_send(int s, const void *buf, size_t len, const ot_ip6 ip, uint16_t port); 44ssize_t socket_send(int s, const void *buf, size_t len, const ot_ip6 ip, uint16_t port);
diff --git a/tests/accept_filter_probe.py b/tests/accept_filter_probe.py
new file mode 100644
index 0000000..22f585a
--- /dev/null
+++ b/tests/accept_filter_probe.py
@@ -0,0 +1,110 @@
1"""What a deferred accept does to connections that never send anything.
2
3Those connections never reach opentracker, so its own client timeout cannot
4clean them up: whatever bound exists is the kernel's. This measures it, and
5then asks the question that follows from it -- whether enough of them keep
6the tracker from answering anybody else.
7
8 make opentracker
9 python3 tests/accept_filter_probe.py
10
11On FreeBSD the filter does nothing unless accf_data is loaded, so run it both
12ways to see whether the numbers differ:
13
14 kldload accf_data
15"""
16
17import os
18import socket
19import signal
20import subprocess
21import sys
22import tempfile
23import time
24import urllib.error
25import urllib.request
26
27TRACKER = os.environ.get("OPENTRACKER", "./opentracker")
28IDLE = int(os.environ.get("IDLE", "300"))
29PATIENCE = float(os.environ.get("PATIENCE", "90"))
30HASH = "".join("%%%02x" % b for b in range(1, 21))
31
32
33def start(port):
34 directory = tempfile.mkdtemp(prefix="ot-accf-")
35 with open(os.path.join(directory, "conf"), "w") as handle:
36 handle.write("listen.tcp 127.0.0.1:%d\nlisten.udp 127.0.0.1:%d\n" % (port, port))
37 log = open(os.path.join(directory, "log"), "w+b")
38 tracker = subprocess.Popen([TRACKER, "-f", os.path.join(directory, "conf")], stdout=log, stderr=log)
39 for _ in range(300):
40 try:
41 socket.create_connection(("127.0.0.1", port), timeout=0.2).close()
42 return tracker, log
43 except OSError:
44 time.sleep(0.02)
45 tracker.kill()
46 raise SystemExit("tracker never listened on %d" % port)
47
48
49def announce(port, timeout=5):
50 """One ordinary request, so we can see whether the tracker still serves."""
51 started = time.time()
52 try:
53 urllib.request.urlopen(
54 "http://127.0.0.1:%d/announce?info_hash=%s&peer_id=-OT0001-probe0000001&port=6881&left=0" % (port, HASH),
55 timeout=timeout,
56 ).read()
57 return time.time() - started, None
58 except Exception as problem: # noqa: BLE001 - reported, not swallowed
59 return time.time() - started, problem
60
61
62def main():
63 port = int(os.environ.get("PORT", "28000"))
64 tracker, log = start(port)
65 try:
66 print(" how long a connection that says nothing is held")
67 quiet = socket.create_connection(("127.0.0.1", port), timeout=5)
68 quiet.settimeout(PATIENCE)
69 started = time.time()
70 try:
71 held = "dropped after %.1f s" % (time.time() - started) if quiet.recv(1) == b"" else "sent us something"
72 except socket.timeout:
73 held = "still open after %.0f s" % PATIENCE
74 except OSError as problem:
75 held = "ended after %.1f s: %s" % (time.time() - started, problem)
76 quiet.close()
77 print(" %s" % held)
78
79 print(" and whether %d of them keep anybody else out" % IDLE)
80 before, problem = announce(port)
81 print(" a request before: %.0f ms%s" % (before * 1000, "" if problem is None else " (%s)" % problem))
82
83 idlers = []
84 for _ in range(IDLE):
85 try:
86 idlers.append(socket.create_connection(("127.0.0.1", port), timeout=2))
87 except OSError as problem:
88 print(" could only open %d of them: %s" % (len(idlers), problem))
89 break
90 print(" holding %d connections that have sent nothing" % len(idlers))
91
92 during, problem = announce(port, timeout=10)
93 print(" a request during: %.0f ms%s" % (during * 1000, "" if problem is None else " REFUSED: %s" % problem))
94
95 for idler in idlers:
96 idler.close()
97 after, problem = announce(port)
98 print(" a request after: %.0f ms%s" % (after * 1000, "" if problem is None else " (%s)" % problem))
99 finally:
100 tracker.send_signal(signal.SIGINT)
101 try:
102 tracker.wait(timeout=10)
103 except subprocess.TimeoutExpired:
104 tracker.kill()
105 tracker.wait()
106 log.close()
107
108
109if __name__ == "__main__":
110 main()
diff --git a/tests/integration/test_expiry.py b/tests/integration/test_expiry.py
index 5a21e57..5e6dc0a 100644
--- a/tests/integration/test_expiry.py
+++ b/tests/integration/test_expiry.py
@@ -27,6 +27,11 @@ class ExpirySweep(TrackerTestCase):
27 def test_the_sweep_takes_the_idle_and_leaves_the_answered(self): 27 def test_the_sweep_takes_the_idle_and_leaves_the_answered(self):
28 idle = socket.create_connection(("127.0.0.1", self.tracker.port), timeout=5) 28 idle = socket.create_connection(("127.0.0.1", self.tracker.port), timeout=5)
29 try: 29 try:
30 # Enough for a kernel that holds a connection back until data
31 # arrives to hand it over, and far too little to be answered, so
32 # the tracker has accepted it and is waiting for the rest
33 idle.sendall(b"GET /ann")
34
30 self.assertEqual(self.tracker.announce(self.hash_for("expiry-sweep"))[b"complete"], 1) 35 self.assertEqual(self.tracker.announce(self.hash_for("expiry-sweep"))[b"complete"], 1)
31 36
32 time.sleep(CLIENT_TIMEOUT + 3 * CHECK_INTERVAL) 37 time.sleep(CLIENT_TIMEOUT + 3 * CHECK_INTERVAL)
@@ -34,6 +39,6 @@ class ExpirySweep(TrackerTestCase):
34 self.assertEqual(self.tracker.torrent_count(), 1, "the tracker stopped serving while sweeping") 39 self.assertEqual(self.tracker.torrent_count(), 1, "the tracker stopped serving while sweeping")
35 40
36 idle.settimeout(5) 41 idle.settimeout(5)
37 self.assertEqual(idle.recv(64), b"", "a connection that never sent a request was not swept") 42 self.assertEqual(idle.recv(64), b"", "a connection that went quiet mid-request was not swept")
38 finally: 43 finally:
39 idle.close() 44 idle.close()
diff --git a/tests/t_event.c b/tests/t_event.c
index ad3948f..706a285 100644
--- a/tests/t_event.c
+++ b/tests/t_event.c
@@ -241,6 +241,28 @@ static void case_close_takes_it_out(void) {
241 pair_close(); 241 pair_close();
242} 242}
243 243
244/* A descriptor can be taken on before anybody waits for anything on it: the
245 full scrape parks a connection while a worker thread builds the reply. The
246 mechanisms disagree about what registering nothing means, so what is
247 checked is that asking for a filter afterwards still works. */
248static void case_add_nothing_then_ask(void) {
249 test_case("a descriptor added with no interest can be given some later");
250 pair_open();
251 CHECK(sv[0] != -1);
252 CHECK_INT(event_add(sv[0], 0, 7), 0);
253 CHECK_INT((int)write(sv[1], "x", 1), 1);
254 CHECK_INT(wait_for(sv[0], EVENT_IN, 200, NULL), 0);
255
256 CHECK_INT(event_modify(sv[0], EVENT_IN, 7), 0);
257 {
258 uint32_t generation = 0;
259 CHECK_INT(wait_for(sv[0], EVENT_IN, 500, &generation), 1);
260 CHECK_INT((int)generation, 7);
261 }
262 event_del(sv[0]);
263 pair_close();
264}
265
244void test_event(void) { 266void test_event(void) {
245 CHECK_INT(event_init(), 0); 267 CHECK_INT(event_init(), 0);
246 CHECK(event_mechanism() != NULL); 268 CHECK(event_mechanism() != NULL);
@@ -254,6 +276,7 @@ void test_event(void) {
254 case_timeout(); 276 case_timeout();
255 case_unknown_descriptor(); 277 case_unknown_descriptor();
256 case_close_takes_it_out(); 278 case_close_takes_it_out();
279 case_add_nothing_then_ask();
257 280
258 event_deinit(); 281 event_deinit();
259} 282}
diff --git a/tests/t_socket.c b/tests/t_socket.c
index ab4b785..20a0c0a 100644
--- a/tests/t_socket.c
+++ b/tests/t_socket.c
@@ -78,15 +78,17 @@ static int sock_flag(int s, int level, int option) {
78} 78}
79 79
80/* Wait for one event, so a test never blocks forever on a lost packet */ 80/* Wait for one event, so a test never blocks forever on a lost packet */
81static int wait_for(int s, short events) { 81static int wait_for_ms(int s, short events, int timeout_ms) {
82 struct pollfd pfd; 82 struct pollfd pfd;
83 83
84 pfd.fd = s; 84 pfd.fd = s;
85 pfd.events = events; 85 pfd.events = events;
86 pfd.revents = 0; 86 pfd.revents = 0;
87 return poll(&pfd, 1, 2000) == 1; 87 return poll(&pfd, 1, timeout_ms) == 1;
88} 88}
89 89
90static int wait_for(int s, short events) { return wait_for_ms(s, events, 2000); }
91
90/* ------------------------------------------------------------ invariants */ 92/* ------------------------------------------------------------ invariants */
91 93
92/* Asked for the wildcard address, because a socket bound to one specific 94/* Asked for the wildcard address, because a socket bound to one specific
@@ -200,6 +202,42 @@ static void case_a_signal_ends_a_blocking_read(void) {
200 close(reader_socket); 202 close(reader_socket);
201} 203}
202 204
205/* What a deferred accept is for is a descriptor that already has the request
206 in it, so what is checked is the silence before: a client that has
207 connected but said nothing must not make the listener readable. Kernels
208 that cannot do this say so, and then there is nothing to check. */
209static void case_deferred_accept(void) {
210 struct sockaddr_in6 address;
211 char ip[16];
212 uint16_t port = 0;
213 int listener, client;
214
215 test_case("a connection that has sent nothing is held back from accept");
216 listener = socket_tcp_listen(ip_loopback, 0, SOCKET_EXCLUSIVE);
217 CHECK(listener != -1);
218 if (socket_defer_accept(listener) == -1) {
219 close(listener);
220 return;
221 }
222 bound_address(listener, ip, &port);
223
224 client = socket(AF_INET6, SOCK_STREAM, 0);
225 CHECK(client != -1);
226 memset(&address, 0, sizeof(address));
227 address.sin6_family = AF_INET6;
228 address.sin6_port = htons(port);
229 memcpy(&address.sin6_addr, ip_loopback, 16);
230 CHECK_INT(connect(client, (struct sockaddr *)&address, sizeof(address)), 0);
231
232 CHECK_INT(wait_for_ms(listener, POLLIN, 300), 0);
233
234 CHECK_INT((int)write(client, "GET / HTTP/1.0\r\n\r\n", 18), 18);
235 CHECK_INT(wait_for(listener, POLLIN), 1);
236
237 close(client);
238 close(listener);
239}
240
203/* An IPv4 peer must arrive in the v4 mapped form opentracker stores, however 241/* An IPv4 peer must arrive in the v4 mapped form opentracker stores, however
204 the kernel chose to report it. Linux fills a v4 mapped sockaddr_in6 here 242 the kernel chose to report it. Linux fills a v4 mapped sockaddr_in6 here
205 and the BSDs fill a plain sockaddr_in, so this is the case that tells the 243 and the BSDs fill a plain sockaddr_in, so this is the case that tells the
@@ -352,6 +390,7 @@ void test_socket(void) {
352 case_a_signal_ends_a_blocking_read(); 390 case_a_signal_ends_a_blocking_read();
353 case_udp_v4mapped(); 391 case_udp_v4mapped();
354 case_tcp_roundtrip(); 392 case_tcp_roundtrip();
393 case_deferred_accept();
355 case_connect_refused(); 394 case_connect_refused();
356 case_bind_failure(); 395 case_bind_failure();
357#ifdef WANT_SYNC_LIVE 396#ifdef WANT_SYNC_LIVE
generated by cgit v1.3 (git 2.53.0) at 2026-10-30 22:51:24 +0000