[1/2]
Incremental HTML Generation
For threads with triple-digit post counts, like this one, the first page load after a new post starts to lag, because the page is rebuilt from scratch. There was a request for some timings in >>119 and a timing method in >>64, but in the intervening months no timings were forthcoming. That's OK, I hacked together my own. Here is the timing method:
$ cat test.scm
(define (timeit proc)
(with-timings proc
(lambda (run-time gc-time real-time)
(write (internal-time/ticks->seconds run-time))
(write-char #\space)
(write (internal-time/ticks->seconds gc-time))
(write-char #\space)
(write (internal-time/ticks->seconds real-time))
(newline))))This diff comments out the server stuff and allows bbs.scm to be partially tested in the REPL of MIT/GNU Scheme 9.1.1. Serve-file and read-file are from deps/server.scm.
$ cat base.diff
--- bbs.scm 2020-06-14 19:41:25.881472281 +0000
+++ bbs-edit.scm 2020-06-14 23:20:48.760044390 +0000
@@ -11,14 +11,24 @@
(load "lib/utils")
(load "deps/irregex")
(load "deps/srfi-26")
-(load "deps/httpio")
-(load "deps/server")
+;(load "deps/httpio")
+;(load "deps/server")
(load "lib/html")
(load "lib/parameters")
(load "lib/markup")
(load "templates")
+(define (serve-file path #!optional headers)
+ (if (default-object? headers) (set! headers '()))
+ (let ((content (read-file path)))
+ `(200 ,headers ,content)))
+
+(define (read-file filename)
+ (call-with-input-file filename
+ (lambda (port)
+ (read-string (char-set) port))))
+
(define (get-form-hash)
"TODO"
(call-with-input-file "hash" read))
@@ -31,7 +41,7 @@
(define (make-abs-path . args)
(string-join (cons "" args) "/"))
-(define server (create-server))
+;(define server (create-server))
(define (make-response template)
`(200 ,(list (make-http-header 'content-type "text/html; charset=utf-8"))
@@ -43,11 +53,11 @@
(make-http-header 'cache-control "Private"))))
;;; static files
-(get server (serve-static "static") '("static"))
+;(get server (serve-static "static") '("static"))
-(get server (lambda (req params) (serve-file "static/favicon.ico")) '("favicon.ico"))
+;(get server (lambda (req params) (serve-file "static/favicon.ico")) '("favicon.ico"))
-(add-handler server (lambda (req params) (route req)))
+;(add-handler server (lambda (req params) (route req)))
(define (ignore-qstring fullpath)
(let ((l (string-split fullpath #\?)))
@@ -414,4 +424,4 @@
(decode-formdata message)
(cdr validation)))))
-(listen server (string->number (car (command-line))))
+;(listen server (string->number (car (command-line))))Here is the generation from scratch for https://textboard.org/sexp/prog/39 at 286, followed by serving from cache:
$ mit-scheme --load bbs-edit.scm --load test.scm
MIT/GNU Scheme running under GNU/Linux
Type `^C' (control-C) followed by `H' to obtain information about interrupts.
Copyright (C) 2011 Massachusetts Institute of Technology
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Image saved on Tuesday February 6, 2018 at 6:31:25 PM
Release 9.1.1 || Microcode 15.3 || Runtime 15.7 || SF 4.41
LIAR/x86-64 4.118 || Edwin 3.116
;Loading "bbs-edit.scm"...
; Loading "format.com"... done
; Loading "lib/utils.scm"... done
; Loading "deps/irregex.scm"... done
; Loading "deps/srfi-26.scm"... done
; Loading "lib/html.scm"... done
; Loading "lib/parameters.scm"... done
; Loading "lib/markup.scm"... done
; Loading "templates.scm"... done
;... done
;Loading "test.scm"... done
1 ]=> (timeit (lambda () (view-thread "prog" "39") 'ok))
1.02 .03 1.055
;Value: ok
1 ]=> (timeit (lambda () (view-thread "prog" "39") 'ok))
.1 0. .092
;Value: okFirst is one second, the other a tenth. Here is a diff for incremental generation. The old cache is renamed on post rather than deleted. Only the new posts go through templating and sxml conversion by controlling the filter, then plain string processing without regex is used to merge the old cache with the new content. The result is identical to the full uncached generation.
Here is a small diff on top of >>292 to time the sexp read/write in bbs.scm:post-message in the REPL of MIT/GNU Scheme 9.1.1. It comments out the thread-list and frontpage updates and makes the redirect absolute.
--- bbs-edit.scm 2020-06-15 22:57:40.125009000 +0000
+++ bbs-edit2.scm 2020-06-17 02:41:55.660698507 +0000
@@ -275,10 +275,10 @@
(content . ,sxml)))))
(call-with-output-file path (lambda (port) (write t port)))
(if (file-exists? cache) (rename-file cache (name-cache-old cache)))
- (if vip
- (update-post-count board thread date post-number)
- (update-thread-list board (string->number thread) date post-number))
- (update-frontpage board)
+ ;(if vip
+ ; (update-post-count board thread date post-number)
+ ; (update-thread-list board (string->number thread) date post-number))
+ ;(update-frontpage board)
(if (equal? frontpage "true")
(redirection board thread (number->string post-number) query-string #t #f)
(redirection board thread (number->string post-number) query-string #f #f))))))
@@ -297,7 +297,7 @@
"That was SICP quality!")
`(303 ,(list (make-http-header
'location
- (string-append (add-query-string (string-append "/" board "/" thread) query-string) "#t" thread "p" post)))
+ (string-append "http://x.y" (add-query-string (string-append "/" board "/" thread) query-string) "#t" thread "p" post)))
"That was SICP quality")))
(define (update-post-count board thread date post-count)The test is on https://textboard.org/sexp/prog/39 at 286. To fake a simple request manually for post-message the mandatory post fields are those that the let* retrieves using a defaultless lookup-def rather than assq.
$ mit-scheme --load bbs-edit2.scm --load test.scm
[...]
;Loading "test.scm"... done
1 ]=> (timeit (lambda () (post-message "prog" "39" (make-http-request "POST" (string->uri "/uri") '(1 . 0) '() "frontpage=false&epistula=hello") "")))
.08 .02 .093
;Value 13: (303 (#[http-header 14 location]) "That was SICP quality")Reading and writing the sexp together take up less than a tenth of a second at 286, so the sexp file does not need the incremental generation treatment.