Lily# The VS Code extension, and what it does with AI

01

Install and open

Installing takes one search in VS Code's Extensions view — Get started walks through it, and through a first score. Open any .lys file and the language server starts, with the score beside it. The keys to know:

KeyDoes
Ctrl+Spacelist what can come next — words, names, chords, marks; in an empty file, the lessons — below
Ctrl+Shift+Vopen the score preview
Ctrl+K Vopen it beside the source
Alt+P (hold)sound the note under the caret
Alt+Mplay the measure the caret is in
Ctrl+Shift+Up / Downstep an octave mark or a chord shape — below
Ctrl+Itransform the selection with AI — below

On macOS read Cmd for Ctrl in the first two and the last; the stepping keys are Ctrl+Shift+Up/Down (the Control key) there too.

02

Typing aids you might not notice

A note is written in one order — pitch, octave marks, duration, dots, then its string number, annotations and marks: cis'4.\2@accent(. The editor knows that order, so a mark typed anywhere on the note lands in its slot, and the caret stays where you pressed the key. You never have to move to the right spot first. In the examples, | is the caret.

Octave marks land after the pitch

Type ' or , with the caret anywhere on a note — at its start, in the middle of its name, after its duration or dots, or among its annotations and marks — and the mark is inserted after the pitch letters and the marks already there, before the duration.

BeforeTypeAfter
c|is4.'c|is'4.
cis4.|'cis'4.|
cis4.@accent|'cis'4.@accent|
c,4|'c4| — the opposite key cancels one mark
<c e g>4|'<c e g>'4| — the whole chord

Marks stack from wherever the caret rests (' twice gives c''4). Inside a chord's brackets the mark is the member's and is typed where you press it (<c e| g> → <c e'| g>); the chord as a whole takes one at its < or after its >. A rest takes no octave mark, so there the key types as pressed.

An octave mark typed in the middle of cis4. lands after the pitch.

A duration digit replaces the duration

Type a digit with the caret anywhere on a note, rest, chord or slash note and it goes to the duration slot, and what it leaves there is always a real duration — 1, 2, 4, 8, 16, 32, 64 or 128. A digit that makes one together with the digits already there extends them; otherwise it replaces them. Dots stay.

BeforeTypeAfter
c4|8c8|
c|48c|8
c16|8c8|
c1|6c16| — 16 is a duration, so the 6 extends
c1|2c2| — 12 is not, so the 2 replaces
c4|6c64| — a 3 or 6 on its own is completed to 32 or 64
c4.|8c8.| — the dot is kept
<c e g>4|8<c e g>8|
r4|2r2| — rests too
/4|8/8| — and slash notes
c4@f|8c8@f|
c4|5c4| — no duration begins with 5: nothing changes

Directly after the digits a digit extends them only when the result is a duration (c1| + 6 → c16, c3| + 2 → c32). Anywhere else on the note a 1, 2, 4 or 8 starts a fresh duration (c1.| + 2 → c2.), while a 3 or 6 — no duration by itself — extends when it can (c1.| + 6 → c16.). A 3 or 6 that extends into nothing is completed to the only duration it can start: 32 or 64. Retyping the duration a note already has changes nothing. A digit that begins no duration — 5, 7, 9, 0 — leaves the note unchanged and says so in the status bar: on a note it could only write an error such as c45.

A 128th cannot be typed digit by digit, since c1| + 2 is already c2: paste 128, type it with the aids turned off, or type 8 after a 12 that is already there. Digits inside a chord's brackets (scale degrees), inside an annotation's parentheses (@finger(5)) and outside notes (time 5/4, R1*5) are typed as pressed.

Typing a digit on a note replaces its duration.

The rest of the slots

KeyWhat happensExample
.the dot goes after the duration and any dots already there (on a rest too)c|'8. → c|'8..
\opens the note's tab string number in its slot and the caret follows, ready for the digit; on a note that already has one, the digit is selected so the next key replaces it|a4 → a4\|
@goes after the string number and the annotations already there, and the name list opens therec|4( → c4@|(
~moves to the end of the note|c2 → c2~
( / )a slur is placed over two notes at once — the one typed on and the next; ) on the note after a slur's end extends it by onec4| d e → c4( d) e
[ / ]a manual beam, the same way, within the barc8| d e f → c8[ d] e f
< / >< before a note wraps it in a chord (the duration stays outside); turning one end into << or back turns the matching end too|c4 → <c|>4

Several cursors. With more than one cursor, the octave marks, digits, dots, \ and @ work at each cursor on its own note, all in one undo step: c4| d1| /4| + 6 → c64| d16| /64|. Two cursors on the same note have no single answer, so there the key is typed as pressed at every cursor. The bracket, slur, beam and tie keys are single-cursor: with several cursors they type as pressed.

When it steps aside. With a selection the key replaces the selection, as anywhere in VS Code. Keys inside strings and comments, and outside notes — time 4/4, tuplet 3/2, a title — type as pressed. A slash note takes durations, dots, ties, slurs and beams like a note but has no pitch, so ', , and \ type as pressed on it, as on a rest. Every aid is one undo step with the keystroke. To turn all of it off, set lilysharp.typingAids.enabled to false (Settings → Lily# → Typing Aids): every key then types exactly as pressed.

03

The preview

The score redraws as you type, not when you save: a keystroke reparses only what it touched, and only the systems whose width changed are laid out again.

  • It opens by itself and follows you. Opening a .lys file opens the preview beside it (lilysharp.preview.openOnOpen); switching to another .lys turns the same preview to it, each file keeping its own zoom and scroll, and while it plays it waits for the music to stop. The lock at the top right of the preview keeps it on one file — to compare two scores, lock one and switch to the other: its preview opens beside the locked one. Bringing a locked preview forward brings its file's tab forward too, and a .lys opened while the preview has the focus opens among the scores, not over the preview.
  • Source and score are linked both ways. The note under the caret is highlighted in the score, and a selection highlights every note in it. Click a note, a tie, a slur, a clef or a key signature in the preview to jump to the text that wrote it.
  • It plays. The ▶ and ⏹ buttons at the top play and stop, from the highlighted note if there is one; the picker beside them chooses which score the preview shows. Right-click for Play from Here, Stop Playback, Fit Width and Reset Zoom. Playback highlights each note as it sounds, with each part's own General MIDI sound, and plays the score the preview shows — that score's parts, under its form.
  • Its sound is a SoundFont synthesizer. Out of the box it plays the General MIDI bank your computer already has; for fuller instruments, install a free sound font — Sound fonts, below. A browser lets sound start only once you have clicked or typed in the preview itself — the status bar says so when a note comes first.
  • It exports. Lily#: Export writes PDF, SVG, PNG, LilyPond (.ly), MusicXML, MIDI or a VOCALOID sequence — from the preview (the score it shows, under that score's own form), the command palette, or by right-clicking .lys files in the Explorer (every score of every selected file, into one folder). Every door writes the same file as lysc.
  • Red rings mark the @todo notes, with their memos in the staff's margin; an export draws none (lysc svg --todo does). lilysharp.preview.highlightTodos hides them.
  • Its colours follow VS Code's theme, or stay light (the printed page) or dark: lilysharp.preview.theme.
The preview follows each keystroke; caret and clicks link source and score.

04

Stepping: Ctrl+Shift+Up / Ctrl+Shift+Down

Two keys that step what the caret is on, one notch at a time.

OnUp / Down
a noteadds one ' (Up) or , (Down) — or takes away one of the other mark. Inside < > or << >> only the member under the caret moves; on the chord's > or its duration, the whole chord moves. With a selection, every note in it steps. It is a plain mark edit, so in relative mode the notes after it follow.
a chord name — @chord(Cm7) or a row's Cm7Up on a name with no shape writes its usual shape and the diagram appears (@chord(Cm7) → @chord(Cm7 x35343), G → G(320003)); further Ups and Downs walk the other playable shapes, frets 10–15 included; Down back at the usual shape removes it. The status bar says where you are: Cm7: shape 4 of 33 (x3x546).
a chord(…) itemsteps its shape the same way on the part's own instrument (from chord(C), Up writes the usual shape); Down stops at the usual shape, since the item needs one.
anything elsenothing on Windows and macOS; on Linux, VS Code's own Add Cursor Above / Below

All cursors step in one undo step. The shapes walked are the playable ones — at least three strings, chord tones only, the root lowest, within four frets and four fingers; stretch shapes (fretted frets five apart) are left out unless "lilysharp.chordShapes.includeStretch": true. Hover a chord to see what it draws, or what Up would write. In a score whose layout says chordDiagrams all a name already draws its usual shape, so there Up writes the next one. What a diagram is and which instrument it is drawn for: Chords and chord diagrams.

To use other keys, open Keyboard Shortcuts (Ctrl+K Ctrl+S) and bind lilysharp.stepUp / lilysharp.stepDown.

Ctrl+Shift+Up/Down on a note, on one member of a chord, and on the whole chord — each result sounds.

05

Audition

With a preview open, edits sound through the preview's synthesizer, with the pitches the compiler plays: the note, chord or @chord shape the caret lands on (once per note, not on every arrow-key repeat), a note as you type it once its pitch and octave marks are in, and the result of every step. Nothing sounds while no preview is open. "lilysharp.audition.enabled": false turns it off; Alt+P and Alt+M keep working either way.

The caret's note sounds as it moves, and a typed note sounds as it is written.

06

Sound fonts

The preview plays each part with the General MIDI instrument its midiInstrument or instrument preset names — the same sounds the exported MIDI asks for. Which recording of those instruments you hear is the sound font.

  • Out of the box Lily# plays the General MIDI bank your computer already has: on Windows the Microsoft GS Wavetable bank (gm.dls, 3 MB), on a Mac the bank of its DLS music device. Every instrument and drum kit is there, but the sound is small and dated. On Linux there is none, and the preview plays simple tones until you choose a font.
  • For fuller sound, download a free General MIDI sound font and choose it. Lily# does not bundle one, so the extension stays small and you pick the sound you like.
Sound fontSizeLicenceCharacter
MuseScore_General.sf336 MBMITfull and stereo; MuseScore 3's default — the one to try first
FluidR3Mono_GM.sf314 MBMITlighter and mono; MuseScore 2's default — for a slower computer
GeneralUser GSabout 30 MBits author's own, freebalanced; download from the author's page

The links are to the fonts' own publishers. More, and their licences, on MuseScore's sound font page; any .sf2, .sf3 or .dls General MIDI font works.

Setting one up

  1. Download the file (for MuseScore_General, click its name above). Keep it in a folder you will not tidy away — Documents\Sound fonts, say. Lily# reads it from there; it does not copy it.
  2. In VS Code, open the command palette (Ctrl+Shift+P, Cmd+Shift+P on a Mac), type sound font and choose Lily#: Choose Sound Font….
  3. Pick Browse for a file… and select the file you downloaded.
  4. The open previews switch to it at once — press ▶ to hear the difference. (A large font can take a moment to load when a preview opens.)

Switching back and forth. Choose Sound Font… remembers the last five fonts you have used and lists them under Default, so trying one against another is a pick, not a browse. A font whose file is not there when you open the list — on a drive that is not plugged in, say — is left out until it is back; the 🗑 button beside a font takes it off the list (the file stays where it is). To go back to the computer's own sounds, pick Default. The choice is the lilysharp.playback.soundFont setting, which holds the file's path.

07

Completion, hover, fixes

  • Lessons. At the top level of a .lys file Ctrl+Space offers lesson-01-completion; type lesson for all twelve — from the completion list and the preview to drums, a band and a jazz chart. Each is a score whose steps are marked in red in the preview (@todo); picking one replaces the file, after asking. The same lessons are on the Learn pages.
  • Completion (Ctrl+Space) offers what can come next — after lyrics NAME it offers sings, then the part names in the file; inside layout { }, fonts { } and paper { } the keys and their values. In a section's music it offers the key's chords, and accepting one writes its notes (IIm7 in C writes <d f a c>); in a chords row or an @chord(…) it offers the key's chord names.
  • Hover names a chord — symbol, degree and pitches (Dm (IIm) D4 F4 A4) — and shows a chord's diagram, or how to add one. Over a phrase's name it shows the phrase's length, how often the file plays it and its source; a drum phrase adds its beat as a grid, one row per drum.
  • Value words complete too: instrument, clef and key offer their words, and midiInstrument " lists the 128 General MIDI sounds.
  • Diagnostics as you type, worded to be acted on, and quick fixes where there is one obvious repair: a section written at different lengths gets Add bar lines (pad the shorter parts) and, when one part has cut it into sections, Split Sections to Match a Part; a name or value that differs from a real one only in case (@upbow, X32010) gets Write '@upBow'; a missing brace or rest can be inserted.
  • Rename a part or a phrase and every reference follows; go to definition, find references, the outline, folding, formatting and semantic colouring work as in any language.
  • A CodeLens over each section says how long it is, who writes it and how often each form plays it — or, when the parts disagree, each length and who writes it.
A section that is not the same length everywhere, fixed from the lightbulb.

08

Commands

In the command palette under Lily#:, and most of them on the editor's right-click menu. Every command that rewrites the file checks the result before it offers it, and applies as one undo step. Each one is described, with examples, on Editor commands.

CommandDoes
New Scoreopens a new .lys from a template, with the title selected
Import MusicXML…turns a .xml / .musicxml / .mxl into a Lily# file — a faithful starting point, with a report of what was approximated
Regroup (by part ⇄ by section)rewrites the file between part bass { section A { … } } and section A { bass { … } }
Split Sections to Match a Partwhen one part has cut a section into several (A of 16 bars + B of 121) and the others still write it whole, cuts the others — and the section's chord rows and lyrics — at the same bars, and makes every form play A B where it played A. If parts subdivide it differently it asks which to follow; one confirmation shows the plan. It is checked first — every cut part must sound and count exactly as before — and a slur, tie or hairpin across a cut is carried into the new section. A beam, a pedal or another span across a cut, or a cut mid-bar, is reported instead and nothing changes. Files grouped by part.
Convert Octaves to Absolute / to Relativerewrites the whole file into the other octave mode, every note at the pitch it sounds, checked note by note
Extract Phrase from Sectionlifts music into a phrase and puts its name in its place — only if it still sounds the same
Add Chord Track (auto-harmonize)adds a chords row that harmonizes the melody with the key's chords, and places it above the staff — a starting point to edit
Guess Section Key…reads the notes of the section at the cursor (or every section the selection touches) and offers its likely keys, best first; the one you pick is written as the key the section starts in, then the notes spelled against it are offered for respelling
Respell Notes to Key…respells the section's notes to the key it starts in (fis → ges in G♭ major, or one spelling where fis and ges are mixed); you choose which respellings to apply, and every note keeps its pitch
Play Note at Cursor · Play Current MeasureAlt+P (hold) · Alt+M
Step Up · Step DownCtrl+Shift+Up · Ctrl+Shift+Down — above
Open Preview · Open Preview to SideCtrl+Shift+V · Ctrl+K V
Export as PDF / SVG / PNG / LilyPond / MusicXML / MIDI / VOCALOIDwrites the file's scores
Transform Selection with AI… · Select AI Model…below
Split Sections to Match a Part, from the length warning to the result.

09

Settings

SettingDefaultDoes
lilysharp.audition.enabledtruesound notes as you edit (Audition)
lilysharp.chordShapes.includeStretchfalselet chord-shape stepping walk stretch shapes too
lilysharp.preview.openOnOpentrueopen the preview beside a .lys file when it is opened (The preview)
lilysharp.preview.highlightTodostruedraw the @todo rings and memos in the preview
lilysharp.playback.soundFontthe computer's ownthe sound font the preview plays with (Lily#: Choose Sound Font…) — which to install, and how
lilysharp.fontPathsmore folders to find SMuFL music fonts in, for fonts { music "…" }
lilysharp.typingAids.enabledtruethe typing aids above (Typing aids)
lilysharp.omr.verifyOnSavefalseon saving a score read from a scan, check it against the scan
lilysharp.preview.themeautothe preview's colours: follow VS Code, light or dark
lilysharp.preview.autoRefresh · refreshDelaytrue · 100 msredraw on every change, after this pause
lilysharp.completion.flatSpellingfullhow completion spells flats (both spellings compile)
lilysharp.import.relativeOctavefalseMusicXML import writes relative octaves instead of absolute
lilysharp.ai.model · ai.ghostModelautomaticthe models the AI features use (AI)
lilysharp.ai.ghostCompletionfalsesuggest the next measure as ghost text
lilysharp.serverPath · trace.serveruse another language server; trace its traffic

10

AI in the editor

The AI features use the language models VS Code offers — GitHub Copilot's, or those of another extension that provides models. Lily# keeps no key or account of its own and has no telemetry; what happened is logged to an output channel on your machine.

Transform Selection with AI — Ctrl+I

Select a few bars — in the text, or by clicking and shift-clicking notes in the preview — press Ctrl+I (Cmd+I) and ask in words: “harmonize this in thirds”, “transpose up a fourth”, “add a crescendo”.

  • The model is given what the compiler knows: the Lily# grammar, the file around the selection, and the file's current errors and warnings.
  • Every candidate is compiled before you see it. One that does not compile, adds warnings, or puts notes more than an octave away from the ones they replace is sent back with the reason, for up to two repairs.
  • You judge the music, not a diff. The candidate is engraved beside the original, with an After/Before toggle. It may change the file around the selection when it has to (a harmony line needs a new part and a score row).
  • Nothing changes until you accept, and one Ctrl+Z undoes it.
Transform Selection with AI: the answer arrives compiled and engraved.

Ghost completion

lilysharp.ai.ghostCompletion (off by default) suggests the next measure as ghost text just after you type a barline; each suggestion is compiled first and shown only if it is valid. It needs a fast model, so it has its own choice, lilysharp.ai.ghostModel. Switching it on offers to enable inline suggestions for .lys files — Lily# keeps them off there by default, because Copilot's own completion slowed every keystroke in a score.

Choosing the model

Lily#: Select AI Model… picks one for good (lilysharp.ai.model). Without a choice, Transform takes the largest model on offer and passes over the small ones, which write poor Lily#; ghost completion takes a small, fast one.

Copilot Chat knows the grammar

The extension gives Copilot the Lily# grammar as an instructions file for .lys files, so Copilot Chat, agent mode and inline chat answer in Lily# instead of guessing LilyPond. (Copilot's as-you-type suggestions do not read instructions files — that is Copilot's rule.)

11

Other assistants and agents

The grammar the extension gives its models is a plain file in the repository, docs/GRAMMAR_FOR_LLM.md: the whole language, its gotchas and its reserved words, written for a model to read. Give it to any assistant that writes .lys, and let the compiler close the loop the way the extension does — lysc check song.lys prints every error and warning with its line and column, and exits non-zero on an error. For generated music, octave absolute keeps a wrong octave from cascading (Pitches and octaves).

There is no MCP server or packaged agent skill; the grammar file and lysc are the interface.

12

Scores in Markdown

A fenced block marked lily# in a Markdown file draws its score in VS Code's built-in Markdown preview, the way a mermaid fence draws a diagram. The fence holds a complete file — an existing .lys pasted as it is — and draws its default score (the unnamed one, else the first), laid out as a snippet: one page as tall as the music.

```lily#
part m { clef treble }
section A { m { c4 d e f | g1 | } }
form { ~A }
score { staff m }
```

13

The lysc command line

The same compiler, for batch work and scripts:

lysc check song.lys         // errors and warnings, nothing written
lysc svg song.lys           // song.svg, plus song-NAME.svg for each named score
lysc svg --score tab song.lys     // only the score named tab
lysc svg --todo lesson.lys        // with the @todo rings and memos
lysc svg --set music=Bravura song.lys   // another music font, over the file's
lysc check --todo-as-error song.lys     // fail while a @todo is left (--no-todo: say nothing)
lysc pdf | png | midi | xml | ly | vsqx song.lys
lysc import score.musicxml  // MusicXML to .lys
lysc octave --absolute song.lys   // rewrite into absolute (or --relative) octaves
lysc harmonize song.lys     // suggest a diatonic chord track for the melody
lysc layout song.lys        // where the lines break, bars per system

-d DIR chooses the output folder, and --batch LIST runs a command over many files in one process. lysc --help lists the rest.