Difficulty	intermediate
Time	An evening
Cost	$0 to $0 (USD)
Skills	JavaScript or TypeScript, reading hexadecimal, patience with 1982 conventions

Parts

Tools

Node.js 20 or later, or any language with byte arrays
A text editor
VICE (x64sc) or another C64 emulator, to test the result

What you are building

A Commodore 64 does not keep BASIC programs as text. When you press RETURN on a line, the interpreter's CRUNCH routine replaces every keyword with a single byte, links the line to the one after it and stores the result in RAM from address 2049 upward. A program saved to disk is that memory image with a two-byte header. If you can produce the same bytes from a text file, you can take any listing printed in a 1984 magazine, or any program typed into a modern editor, and load it straight into an emulator or a real machine over an SD2IEC. This site builds its type-in programs exactly this way; the converter described here is the one in its repository, and the steps below reproduce it from nothing.

The file format

A .prg file begins with the load address, low byte first. For a BASIC program that is 01 08, meaning 2049. After the header come the lines, each with the same shape:

Two bytes: the address of the next line, low byte first. The machine uses these as a linked list and ignores them on load, but they must be right, because LIST walks them.
Two bytes: the line number, low byte first.
The line's contents, with keywords tokenised and everything else as PETSCII.
One zero byte to end the line.

After the last line come two more zero bytes: a next-line pointer of 0 marks the end of the program. So the smallest possible program, a single line 10 END, is 01 08, then 09 08 (the next line would start at 2057), then 0A 00 for line 10, then 80 for END, then 00, then 00 00. Twelve bytes.

The line links are the part people get wrong. Each one is the address where the following line begins, which is the current line's address plus 4 (link and number), plus the length of its contents, plus 1 for the terminator. Compute them as you go, starting from 2049.

The token table

Commodore BASIC V2 has 76 keywords, numbered from 128 upward in a fixed order, and the number pi has the token 255. The order matters because it is also the order the interpreter searches, and your converter must search it the same way. The first sixteen are END, FOR, NEXT, DATA, INPUT#, INPUT, DIM, READ, LET, GOTO, RUN, IF, RESTORE, GOSUB, RETURN and REM, taking values 128 to 143; the table continues through the statements, then the operators (+ is 170, - is 171, the comparison operators 177 to 179), then the functions, ending with MID$ at 202 and GO at 203. C64-Wiki prints the full table; copy it into your code as an array indexed from 128 and check it twice, because a one-position slip produces a program that loads, lists plausibly and does something else.

Two entries deserve attention. INPUT# comes before INPUT, and PRINT# before PRINT, so a search that stops at the first match from the top of the table finds the longer keyword first. GOTO is early in the table and GO is last, so GOTO becomes one token while GO TO becomes GO, a space, and TO. The question mark is not in the table but the ROM treats it as PRINT, and so must you.

The three rules

CRUNCH walks along the typed line and, at every position, tries the keywords in table order. When one matches it writes the token and skips past the keyword; otherwise it copies one character and moves on. Three exceptions make the difference between a converter that works and one that corrupts strings.

Inside quotation marks nothing is tokenised. A quote flips a flag; while the flag is set, characters are copied as they are. PRINT "FOR" stores the word FOR as three letters.
After REM the rest of the line is copied literally. The remark token is written, then everything to the end of the line, keywords included.
After DATA, characters are copied literally up to the next colon. DATA FOR,TO stores the words as text; the colon ends the literal run and tokenising resumes.

Everything else is tokenised, including inside variable names, which is why a C64 program cannot have a variable called TOTAL: the interpreter stores it as TO followed by TAL. If a listing from a magazine seems to have an odd variable name, this is usually why.

Letters and control characters

The text in a listing has to become PETSCII, and the case convention is the reverse of what you expect. On a C64 in its default character set, the letters you see as capitals are the unshifted codes 65 to 90. VICE's petcat tool, which converts listings in both directions, writes those as lower-case letters in the text file and uses upper-case letters for the shifted codes 193 to 218, which appear as graphics symbols in the default set and as capitals in the alternate one. Follow the same convention and your text files will match what petcat produces, which makes testing easy.

Control characters are the other half of the problem. A listing that clears the screen with PRINT CHR$(147) is fine, but most magazine listings embedded the control character inside a string, where it showed as a reverse-video heart. petcat writes these as names in braces: {clr} for 147, {home} for 19, {wht} for 5, {rvon} for 18, {down} for 17 and so on. Expand the names to their byte values when you meet them inside a string, and accept {$93} style hex escapes for anything without a name.

Writing it

In TypeScript the whole converter fits in a screen or two. The core is a function that takes one line of text, strips the line number, then loops over the remaining characters with three booleans for the quote, REM and DATA states. At each position it first checks for a brace escape, then for a quote, then for the literal states, then for a question mark, then tries the token table against the upper-cased remainder of the line, and finally copies a single character converted to PETSCII. It returns the line number and an array of bytes.

A second function assembles the lines: it starts an address counter at 2049, and for each line writes the link, the number, the bytes and a zero, advancing the counter by their total length. It finishes with two zeros and prepends the 01 08 header.

Write the detokeniser too. It is simpler, since it only has to replace bytes from 128 upward with the keyword text while respecting quotes, and it gives you a round-trip test: convert a listing to bytes, convert the bytes back, and compare. The listing that opens every C64 manual is the right first test: 10 PRINT "HELLO":GOTO 10 should produce exactly 01 08 14 08 0A 00 99 20 22 48 45 4C 4C 4F 22 3A 89 20 31 30 00 00 00. If your output differs by one byte in the link, you forgot the terminator in the length.

Testing in VICE

Save the bytes as a file ending in .prg and start the emulator with it on the command line: x64sc -autostart listing.prg. The emulator loads and runs it. Type LIST and compare the screen with the listing you started from. Three failures are common. A SYNTAX ERROR on a line that looks right usually means a keyword was tokenised inside a string, which means the quote flag is wrong. A listing where capital letters have turned into graphics symbols means the case convention is reversed. A program that stops early with no error means a line link points past the end of the program; recompute the lengths.

If you have petcat, which ships with VICE, you can compare directly: petcat -w2 -o reference.prg -- listing.txt produces Commodore's bytes from the same text, and cmp will tell you the first byte where you differ.

Where to take it

The converter in this site's repository adds a few things worth copying: a check that no line exceeds 80 characters, because a real C64 cannot enter a longer one; a check that line numbers rise and stay below 64000; and a fallback that writes unknown control names as escapes rather than failing. From there the natural next step is a tokeniser for the PET and VIC-20, which use the same table at different load addresses, and for BASIC 7.0 on the C128, which extends the table with two-byte tokens. That is a bigger evening.