Keyboard input¶

Info

See the full example source code on GitHub.

Important

This example requires additional third-party packages. Install them with:

From PyPI From source

josephine@laptop:~$ pip install supriya[examples]

josephine@laptop:~$ git clone https://github.com/supriya-project/supriya.git
josephine@laptop:~$ cd supriya
josephine@laptop:~/supriya$ pip install -e .[examples]

Let’s introduce some interactivity.

We can play notes with Supriya using a MIDI keyboard. Don’t have a MIDI keyboard available? We’ll also support using your computer’s QWERTY keyboard just like you can in Ableton Live.

Note events¶

What’s a “note”, anyways?

MIDI represents notes not as durated objects positioned on a timeline, but instead as pairs of “on” and “off” events. The note starts when an instrument handles a note “on” event, and it stops when that instrument handles the corresponding note “off” event. We won’t recreate MIDI here exactly, but we need some simple classes to represent the idea of starting a note with a given pitch (or note number) and loudness (or velocity) and stopping a note at a given pitch. We can do this with a pair of dataclasses:

@dataclasses.dataclass
class NoteOn:
    """
    A note on event.
    """

    note_number: int
    velocity: int

@dataclasses.dataclass
class NoteOff:
    """
    A note off event.
    """

    note_number: int

The integer values for note_number and velocity will range from 0 to 127, just like with MIDI. It’s an obvious choice for this examples because we’ll literally be using MIDI as an input later on. Massaging all input values into a consistent 0-127 range will remove the need for special-casing.

Note

We’ll use dataclasses pervasively in the examples because 1) they save vertical space by removing boilerplate, and 2) their requirement to use type hints (in my opinion) aids legibility. Importantly, we don’t need to write initializers for them.

Managing polyphony¶

While “note” events represent only the start of stop of a note, we need to translate these events into the creation and destruction of synths on the SuperCollider server, and that requires tracking state. We’ll do this with a polyphony manager. Our implementation looks like this:

@dataclasses.dataclass
class PolyphonyManager:
    """
    A polyphony manager.

    Translates :py:class:`NoteOn` or :py:class:`NoteOff` events into actions
    against a :py:class:`~supriya.contexts.core.Context`.
    """

    # the server to act on
    server: supriya.Context
    # a dictionary of MIDI note numbers to synths
    notes: dict[int, supriya.Synth] = dataclasses.field(default_factory=dict)
    # a synthdef to use when making new synths
    synthdef: supriya.SynthDef = dataclasses.field(default=supriya.default)
    # target node to add relative to
    target_node: supriya.Node | None = None
    # add action to use
    add_action: supriya.AddAction = supriya.AddAction.ADD_TO_HEAD

    def free_all(self) -> None:
        """
        Free all currently playing :py:class:`~supriya.contexts.entities.Synth`
        instances.
        """
        with self.server.at():
            for synth in self.notes.values():
                synth.free()

    def perform(self, event: NoteOn | NoteOff) -> None:
        """
        Perform a :py:class:`NoteOn` or :py:class:`NoteOff` event.
        """
        # if we're starting a note ...
        if isinstance(event, NoteOn):
            # bail if we already started this note
            if event.note_number in self.notes:
                return
            print(f"Performing {event}")
            # convert MIDI 0-127 to frequency in Hertz
            frequency = supriya.conversions.midi_note_number_to_frequency(
                event.note_number
            )
            # convert MIDI 0-127 to amplitude
            amplitude = supriya.conversions.midi_velocity_to_amplitude(event.velocity)
            # create a synth and store a reference by MIDI note number in the
            # dictionary ...
            self.notes[event.note_number] = self.server.add_synth(
                add_action=self.add_action,
                amplitude=amplitude,
                frequency=frequency,
                synthdef=self.synthdef,
                target_node=self.target_node,
            )
        # if we're stopping a note ...
        elif isinstance(event, NoteOff):
            # bail if we already stopped this note:
            if event.note_number not in self.notes:
                return
            print(f"Performing {event}")
            # pop the synth out of the dictionary and free it ...
            self.notes.pop(event.note_number).free()

The polyphony manager’s main work is in its perform() method, translating note on and note off events into actions to create or destroy synths on the SuperCollider server. When creating synths, we need to translate the MIDI 0-127 range into Hertz frequency and linear amplitude. We’ll use some conversion helpers to do this. The manager stores state about the notes and synths in a dictionary, mapping note numbers to synth instances so the synths can be freed later, when it handles note “off” events.

The manager also implements a free_all() method to free all the synths at once. We’ll see both perform() and free_all() in use later on.

Handling input¶

Now let’s build some input handlers.

We have some basic requirements for any input handler:

It needs to translate raw input into NoteOn and NoteOff events.
It needs to start up and listen continuously in the background in a non-blocking way.
It needs to be decoupled from the rest of the system. It shouldn’t know about our PolyphonyManager, only that it can pass the generated NoteOn and NoteOff events along to some other callable, configured when the input handler is instantiated.

We’ll define a simple base class for our MIDI and QWERTY input handlers, mainly to ease type hints. Anytime you encounter a setup / do something / teardown pattern, a context manager is an obvious choice, so we’ll use Python’s contextlib.contextmanager() to decorate our listen() stub method:

@dataclasses.dataclass
class InputHandler:
    """
    Base class for input handlers.
    """

    @contextlib.contextmanager
    def listen(
        self, callback: Callable[[NoteOn | NoteOff], None]
    ) -> Generator[None, None, None]:
        # subclasses must implement this method!
        # 1) start the handler's listener
        # 2) yield to the with block body
        # 3) stop the handler's listener
        raise NotImplementedError

She doesn’t actually do anything, so let’s define some concrete implementations.

Handling MIDI¶

MIDI input is relatively simple to setup, at least compared to QWERTY. We’ll use the python-rtmidi library to listen to MIDI messages from attached hardware. Let’s take a look:

@dataclasses.dataclass
class MidiHandler(InputHandler):
    """
    A MIDI input handler.
    """

    port: int | str

    @contextlib.contextmanager
    def listen(
        self, callback: Callable[[NoteOn | NoteOff], None]
    ) -> Generator[None, None, None]:
        """
        Context manager for listening to MIDI input events.
        """
        self.midi_input = rtmidi.MidiIn()  # create the MIDI input
        # set the MIDI event callback to this class's __call__
        self.midi_input.set_callback(functools.partial(self.handle, callback))
        self.midi_input.open_port(self.port)  # open the port for listening
        print("Listening for MIDI keyboard events ...")  # let the user know
        yield  # yield to the with block body
        self.midi_input.close_port()  # close the port

    def handle(
        self,
        callback: Callable[[NoteOn | NoteOff], None],
        event: tuple[tuple[int, int, int], float],
        *args,
    ) -> None:
        """
        Handle a MIDI input event.
        """
        print(f"MIDI received: {event}")
        # the raw MIDI event is a 2-tuple of MIDI data and time delta, so
        # unpack it, keep the data and discard the time delta ...
        data, _ = event
        if data[0] == rtmidi.midiconstants.NOTE_ON:  # if we received a note-on ...
            # grab the note number and velocity
            _, note_number, velocity = data
            # perform a "note on" event
            callback(NoteOn(note_number=note_number, velocity=velocity))
        elif data[0] == rtmidi.midiconstants.NOTE_OFF:  # if we received a note-off ...
            # grab the note number
            _, note_number, _ = data
            # perform a "note off" event
            callback(NoteOff(note_number=note_number))

The listen() context manager creates an rtmidi.MidiIn instance, points its callback at the MidiHandler’s handle() method, opens a port and then yields. When it’s done yielding it just closes the port. Look closer at how we treat the callback passed into listen() . Rather than store a reference to the callback function on the handler instance, we use functools.partial() to “freeze” part of the call to the handler’s handle() method. This way, our custom callback will always be the first argument to listen() as long as we’re inside its context block.

MIDI note on and note off messages arrive into our handle() method from rtmidi as integer triples. The first integer (the status) tells us what kind of MIDI message it is. For note on and note off messages the second integer represents the note number, from 0 to 127 (more than enough to represent a grand piano’s keyboard, which should be good enough for anyone, right?) and the last integer represents the velocity (how fast the key was pressed, which by convention maps to volume level).

Our handle() method checks the first integer to determine if it’s handling a note on or note off, ignoring other types of messages, and then wraps the data into our NoteOn and NoteOff dataclasses, calling the callback callable against them.

Handling QWERTY¶

Input from a computer keyboard is more complicated because we need to simulate a larger pitch space than we have physical keys available. We’ll use the pynput library to listen for key presses and releases, and mimic Ableton Live’s QWERTY keyboard feature. Take a look and then let’s discuss.

@dataclasses.dataclass
class QwertyHandler(InputHandler):
    """
    A QWERTY input handler.
    """

    octave: int = 5
    presses_to_note_numbers: dict[str, int] = dataclasses.field(default_factory=dict)

    @contextlib.contextmanager
    def listen(
        self, callback: Callable[[NoteOn | NoteOff], None]
    ) -> Generator[None, None, None]:
        """
        Context manager for listening to QWERTY input events.
        """
        # setup the QWERTY keybord listener
        self.listener = pynput.keyboard.Listener(
            on_press=functools.partial(self.on_press, callback),
            on_release=functools.partial(self.on_release, callback),
        )
        self.listener.start()  # start the listener
        print("Listening for QWERTY keyboard events ...")  # let the user know
        yield  # yield to the with block body
        self.listener.stop()  # stop the listener

    @staticmethod
    def qwerty_key_to_pitch_number(key: str) -> int | None:
        """
        Translate a QWERTY key event into a pitch number.
        """
        # dict lookups are faster, but this is soooo much shorter
        try:
            return "awsedftgyhujkolp;'".index(key)
        except ValueError:
            return None

    def on_press(
        self,
        callback: Callable[[NoteOn | NoteOff], None],
        key: pynput.keyboard.Key | pynput.keyboard.KeyCode | None,
    ) -> None:
        """
        Handle a QWERTY key press.
        """
        if not isinstance(key, pynput.keyboard.KeyCode):
            return  # bail if we didn't get a keycode object
        print(f"QWERTY pressed: {key.char}")
        if key.char is None:
            return
        if key.char == "z":  # decrement our octave setting
            self.octave = max(self.octave - 1, 0)
            return
        if key.char == "x":  # increment our octave setting
            self.octave = min(self.octave + 1, 10)
            return
        if key in self.presses_to_note_numbers:
            return  # already pressed
        if (pitch := self.qwerty_key_to_pitch_number(key.char)) is None:
            return  # not a valid key, ignore it
        # calculate the note number from the pitch and octave
        note_number = pitch + self.octave * 12
        # QWERTY keyboards aren't pressure-sensitive, so let's create a random
        # velocity to simulate expressivity
        velocity = random.randint(32, 128)
        # stash the note number with the key for releasing later
        # so that changing the octave doesn't prevent releasing
        self.presses_to_note_numbers[key.char] = note_number
        # perform a "note on" event
        callback(NoteOn(note_number=note_number, velocity=velocity))

    def on_release(
        self,
        callback: Callable[[NoteOn | NoteOff], None],
        key: pynput.keyboard.Key | pynput.keyboard.KeyCode | None,
    ) -> None:
        """
        Handle a QWERTY key release.
        """
        if not isinstance(key, pynput.keyboard.KeyCode):
            return  # bail if we didn't get a keycode object
        print(f"QWERTY released: {key.char}")
        # bail if the key isn't currently held down
        if key.char not in self.presses_to_note_numbers:
            return
        # grab the note number out of the stash
        note_number = self.presses_to_note_numbers.pop(key.char)
        # perform a "note off" event
        callback(NoteOff(note_number=note_number))

Like the MidiHandler, the listen() context manager is pretty simple. We create a pynput.keyboard.Listener, point it as two different callback methods (one for pressing keys, and another for releasing them ), start it, yield to the with block, then stop it when we’re done. Easy.

Handling those key presses and releases is tricky though. We’ll treat asdfghjkl;' as our white keys with a mapping to a C5 on a piano keyboard and ; mapping to an F6, and we tyu op as black keys starting from C#5. That’s about an octave and a half of notes when we’d like to simulate the ten-plus octaves available on MIDI. To do this, we’ll store the current octave on the handler, and use the z and x keys to decrement or increment the octave, adding it to the note number mapped from the other keys.

That’s great, but what happens if you change the octave when you’re in the middle of changing the note? When releasing the key, we’d send a NoteOff event corresponding to a different pitch than the original NoteOn event, resulting in “stuck” synths. To fix this, we’ll add state to the MidiHandler, storing the note numbers calculated at the moment each QWERTY key was pressed. When we release the QWERTY key, rather than re-calculate what the note number should be based on the current octave, we look up the note number calculated at key press and issue a NoteOff for that instead.

Integration¶

Now we need to stitch together our input handlers, the polyphony manager, and a server so we can actually hear something. We’ll write one run() function with a number of simple nested functions for the various callbacks our application requires:

def run(input_handler: InputHandler) -> None:
    """
    Run the script.
    """

    def on_boot(*args) -> None:  # run this during server.boot()
        server.add_synthdefs(polyphony.synthdef)  # add the polyphony's synthdef
        server.sync()  # wait for the synthdef to load before moving on

    def on_quitting(*args) -> None:  # run this during server.quit()
        polyphony.free_all()  # free all the synths
        time.sleep(0.5)  # wait for them to fade out before moving on

    def signal_handler(*args) -> None:
        exit_future.set_result(True)  # set the exit future flag

    def input_callback(event: NoteOn | NoteOff) -> None:
        # just play the event via polyphony directly
        polyphony.perform(event)

    # create a future we can wait on to quit the script
    exit_future: concurrent.futures.Future[bool] = concurrent.futures.Future()
    # create a server and polyphony manager
    server = supriya.Server()
    polyphony = PolyphonyManager(server=server)
    # setup lifecycle callbacks
    server.register_lifecycle_callback("BOOTED", on_boot)
    server.register_lifecycle_callback("QUITTING", on_quitting)
    # hook up Ctrl-C so we can gracefully shutdown the server
    signal.signal(signal.SIGINT, signal_handler)
    # boot the server and let the user know we're ready to play
    server.boot()
    print("Server online. Press Ctrl-C to exit.")
    # turn on the input handler and teach it to callback against the polyphony manager
    with input_handler.listen(callback=input_callback):
        exit_future.result()  # wait for Ctrl-C
    # stop the input handler and quit the server
    server.quit()

A few notes about our run() function:

It takes an InputHandler as its only argument, which means we can use the MidiHandler and QwertyHandler classes interchangeably. It doesn’t know anything about their specific implementations, just that they have a listen() context manager method that accepts a callback callable.
The InputHandler’s context block simply waits on a concurrent.futures.Future to resolve, nothing more. When the future resolves, we exit the context block and quit the server.
We add setup and teardown logic to our server via lifecycle event callbacks. Setup involves loading the SynthDef we’ll use to create synths, and to wait for it to load before proceeding. Teardown involves freeing any synths still playing after we stop listening for input events, and then waiting for them to completely fade out before moving on.
Without getting too deep into systems programming, we’ll use a signal.signal() callback to listen for Ctrl-C presses on the keyboard. When you press Ctrl-C to end the script, this callback will set the value of the exit future our script is waiting on, allowing it to progress to its shutdown logic.

Scripting¶

Let’s put final touches on our script so we can run it from the command-line.

We’ll use argparse to define the flags we can pass to the script, with a required mutually exclusive argument group forcing us to pick at least one flag:

def parse_args(args: list[str] | None = None) -> argparse.Namespace:
    """
    Parse CLI arguments.
    """
    parser = argparse.ArgumentParser(
        description="Play notes via your QWERTY or MIDI keyboards"
    )
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument(
        "--list-midi-inputs", action="store_true", help="list available MIDI inputs"
    )
    group.add_argument(
        "--use-midi", help="play via MIDI keyboard", type=int, metavar="PORT_NUMBER"
    )
    group.add_argument(
        "--use-qwerty", action="store_true", help="play via QWERTY keyboard"
    )
    return parser.parse_args(args)

Then we just parse the arguments, check which flag was set, create the appropriate InputHandler and then call run() with it:

def main(args: list[str] | None = None) -> None:
    """
    The example entry-point function.
    """
    parsed_args = parse_args(args)
    if parsed_args.list_midi_inputs:
        # print out available MIDI input ports
        rtmidi.midiutil.list_input_ports()
    elif parsed_args.use_midi is not None:
        run(MidiHandler(port=parsed_args.use_midi))
    elif parsed_args.use_qwerty:
        run(QwertyHandler())

Of course, we might not know what port to use when playing with MIDI, so we’ll use the --list-midi-inputs flag to print out available MIDI port numbers you can pass to the script when running it with --use-midi.

Invocation¶

You can invoke the script with …

josephine@laptop:~/supriya$ python -m examples.keyboard_input --help
usage: __main__.py [-h] (--list-midi-inputs | --use-midi PORT_NUMBER |
                   --use-qwerty)

Play notes via your QWERTY or MIDI keyboards

options:
  -h, --help            show this help message and exit
  --list-midi-inputs    list available MIDI inputs
  --use-midi PORT_NUMBER
                        play via MIDI keyboard
  --use-qwerty          play via QWERTY keyboard

… and you’ll see the options necessary to properly run it.