Answers arrive truncated, stop mid-sentence, or sit blank for twenty seconds and then appear complete. All three are the same problem: something between your server and the visitor is buffering a response that is meant to stream.
Why streaming is fragile
Sohay streams replies word by word, using Server-Sent Events, so the visitor sees an answer forming rather than a spinner.
That requires every hop to pass bytes through as they arrive. Most infrastructure is built to do the opposite — collect a whole response, then send it — which is right for a web page and wrong for a stream.
What Sohay already does
The plugin sets the headers that ask intermediaries not to buffer:
Cache-Control: no-cacheX-Accel-Buffering: no— the header nginx honours- and it clears PHP’s own output buffers
On a plain host that is enough. Some CDNs ignore it.
Fixing it at the CDN
Two settings, both on the chat REST route:
- Bypass the cache for it. A streaming endpoint should never be cached, and a CDN that is caching it will also be buffering it.
- Disable transformations on the response — minification, HTML post-processing, script injection. Anything that rewrites a body has to hold the whole body first.
On Cloudflare that is a cache rule plus turning off the relevant speed optimisations for that path. Other CDNs have equivalents under different names.
If it is not the CDN
Other things that buffer, in rough order of likelihood:
- Server-level page caching on the host — some managed WordPress hosts buffer at the edge before any CDN is involved.
- A security or firewall plugin inspecting responses.
- An
output_bufferingvalue set in PHP at the server level. mod_deflateor gzip compression applied to the streaming response.
The quickest way to tell which layer is responsible: try the chat with the CDN bypassed, on the origin directly. If it streams there, the problem is in front of your server. If it does not, it is on it.
Checking the errors
Sohay → Diagnostics shows the last 100 log entries, and a stream that failed rather than one that was merely slow will usually have left something there.
A related symptom
If answers are fine but some visitors are turned away entirely, that is more likely a rate limit than a buffering problem — and behind a proxy, one that is misfiring because every visitor looks like the same IP. See Daily token caps and rate limits, which covers the proxy-aware setting and its security caveat.
