repltest.scrbl (4758B)
1 #lang scribble/manual 2 @require[@for-label[repltest 3 racket/base] 4 scriblib/footnote] 5 6 @title{REPL test: copy-paste REPL interactions to define tests} 7 @author[@author+email["Suzanne Soy" "racket@suzanne.soy"]] 8 9 Source code: @url{https://github.com/jsmaniac/repltest} 10 11 @defmodulelang[repltest]{ 12 The @racketmodname[repltest] language is a meta-language 13 that replays a copy-pasted transcript of an interactive 14 REPL (@racket[read-eval-print-loop]) session, checking that the 15 outputs have not changed. 16 17 This allows to quickly write preliminary unit tests based 18 on a debugging session. It is however not a substitute for 19 writing real tests, and these tests are more prone to the 20 “copy-pasted bogus output into the tests” problem.} 21 22 @racketblock[ 23 @#,hash-lang[] @#,racketmodname[repltest] @#,racketmodname[racket] 24 (define x 3) 25 @#,racketid[>] (+ x 1) 26 @#,racketresultfont{4} 27 ] 28 29 The first part of the file is kept inside the top-level 30 module. This module uses the language indicated just after 31 @racket[@#,hash-lang[] @#,racketmodname[repltest]], for 32 example: 33 34 @racketblock[ 35 @#,hash-lang[] @#,racketmodname[repltest] @#,racketmodname[typed/racket]] 36 37 After the first occurrence of the prompt (by default 38 @racket["> "], later versions of this package will allow 39 customizing this) is encountered, all the remaining contents 40 of the file are understood as a REPL transcript. The prompt 41 is only recognized if it is outside of any s-expression, 42 which means that the @racket[>] function can be used 43 normally. 44 45 @racketblock[ 46 @#,hash-lang[] @#,racketmodname[repltest] @#,racketmodname[racket] 47 (define x (> 3 4)) 48 @#,racketid[>] x 49 @#,racketresultfont{#f} 50 ] 51 52 @section{The @racketid[test] submodule} 53 54 This language injects a @racketid[test] submodule 55 using @racket[module*]. When the @racketid[test] module 56 is run, the expression after each prompt is read and 57 evaluated as if it had been typed inside a real REPL, using 58 @racket[read-eval-print-loop]. The result, as printed on the 59 standard output and standard error, is compared with the 60 text read until the next prompt. The next prompt will only 61 be recognized if it is not part of an s-expression, which 62 means that occurrences of @racket[>] inside an expression in 63 the output are correctly handled: 64 65 @racketblock[ 66 @#,hash-lang[] @#,racketmodname[repltest] @#,racketmodname[racket] 67 (define x '(> 3 4)) 68 @#,racketid[>] x 69 @#,racketresultfont{'(> 3 4)} 70 @#,racketid[>] '(> 5 6) 71 @#,racketresultfont{'(> 5 6)} 72 ] 73 74 The fact that a real REPL is used means that any 75 language-specific output will be produced as expected. For 76 example @racketmodname[typed/racket] prints the type of the 77 result before the result itself, so it must be included in 78 the expected output: 79 80 @racketblock[ 81 @#,hash-lang[] @#,racketmodname[repltest] @#,racketmodname[typed/racket] 82 (define x 0) 83 @#,racketid[>] x 84 @#,racketoutput{- : Integer [more precisely: Zero]} 85 @#,racketresultfont{0} 86 ] 87 88 @section{Warning concerning comments} 89 90 Comments are not currently supported inside the REPL 91 transcript. Also, the current version does not recognise the 92 first prompt if it is preceded by a comment. 93 94 @section{Warning concerning spaces and newlines} 95 96 The tests are space-sensitive, so care should be taken to 97 include a newline at the end of the file. This is due to 98 the fact that in most languages, the REPL prints a newline 99 after the result. Furthermore, extra spacing like blank 100 lines should not be added in the transcript part of the 101 file. 102 103 @section{Future improvements} 104 105 Later versions of this package will allow customizing the following aspects: 106 107 @itemlist[ 108 @item{Flexibility of whitespace comparisons (strip leading 109 and trailing whitespace, or ignore all whitespace 110 differences).} 111 @item{Support comments before and inside the REPL 112 transcript.} 113 @item{Specifying a regexp matching the prompt, and a 114 regexp for characters preceding the prompt which are not 115 part of it (and therefore will be part of the preceding 116 result or main module's code).} 117 @item{Disable calling @racket[read] on the output 118 expressions, which can be useful when the output contains 119 unbalanced parenthesis, or do not otherwise match the 120 language's syntax, for example: 121 122 @; TODO: include this in the tests 123 @racketblock[ 124 @#,hash-lang[] @#,racketmodname[repltest] @#,racketmodname[racket] 125 @#,racketid[>] (displayln "(unbalanced") 126 @#,racketresultfont{(unbalanced} 127 @#,racketid[>] (displayln "#invalid (syntax . too . many . dots)") 128 @#,racketresultfont{#invalid (syntax . too . many . dots)}] 129 130 This will also have the side-effect of allowing the prompt 131 to be matched inside s-expressions.} 132 @item{Distinguish standard output (purple font in DrRacket), 133 printed result (blue font) and standard error (red font).}]