creating the aetheryte radio

sometimes when i work i like to listen to ambiance. one video in particular is the "aetheryte asmr", which is an in game recording of the aetheryte crystal. for the past 6 years i've been listening to the same youtube video on repeat. there are a number of problems with this approach:

  1. the youtube player's "loop" functionality doesn't always work. it might loop 20 or 200 times before failing(?) and auto-playing another video. this is jarring.
  2. the audio in the video is a ~7 minute recording, but the loop isn't seamless (i spent far too long re-architecting my music player's internals to support gapless playback), and while it's long enough that i don't care when it cuts and loops, it would be nice if it were seamless.
  3. of course i can't keep the game open.. that's far too resource intensive, other players (ffxiv is an mmorpg) could approach nearby and create noise and ruin the effect, etc.

none of these were deal breakers for me. 6 years is a pretty long time to go facing some struggles before sitting down and deciding to tackle a problem.

nevertheless, one day i finally sat down and thought to myself.. "how can i get the best experience for the ambience without sacrifice?"

the first answer is that i needed to get the game assets. whatever the game was using to play the noise, i needed to extract. with those in hand, i could recreate or play with them in whichever way i wanted.

getting the source assets

ffxiv isn't a super well protected game (at least i don't think.) if you web search "ffxiv data explorer" there are old java apps that load up the asset packages from the games install directory and let you export (presumably) compressed assets. i don't remember which fork i used, but i do know that it came with a hash table that mapped the file to an english file name.

you're going to laugh at me (and honestly that's ok.) i spent 4 hours coming through 4000 audio assets trying to find the audio assets. i literally ctrl+a'd all 4000 .wav files and pressed the left and right arrow keys to move back and forth within the large mpv playlist, looking for the aetheryte noise. the funny part is that i didn't find them, and had to go to sleep at 3am.

i'm an engineer though, right? work smarter, not harder (why didn't i do this before doing the tedious work? idk.) was there any better way to get the sound? did i miss it during my 3am comb over? (hint. i did.) turns out, there is! ffxiv has an extensive modding community and platform, and while i don't play the game much anymore, it was exhilarating to see how much effort people put into customizing their experience.

xivlauncher, vfxeditor, and soundfilter

if you boot ffxiv using xivlauncher (endorsed by my roommate who has 180 days of play time) you can load up plugins. one plugin of interest was soundfilter. i know the name is a bit misleading, but it has the ability to display the names of now playing assets. this helped a ton because i was able to teleport to a new area in the game (the aetherytes are basically waypoint / teleport crystals) and figure out which sound was being played. quickly after i found that the asset i was looking for was bgcommon/sound/fst/placednpc_ethelight_big_loop.scd/0. i quickly exported said asset and was on my way.

an interesting note about that asset is the /0 at the end. i'm pretty sure the sound "asset" contains multiple sounds (i think they were called tracks in game.) 0 was simply the "hum" or the low frequency of ambiance.

you can preview the assets here beware! these are loud. i did not normalize these to preserve integrity
hum
whir 1
whir 2

sadly, getting the actual ambiance would require a bit more elbow grease. i could have just thrown these together in logic pro and called it a day, but where's the fun in that? plus, i'd have to create an excessively long track. i didn't want to be subconsciously aware of when the loop repeats. i wasn't too worried about this though, as i had the right tools up my sleeve to solve my first world made up problem.

web audio api

enter the web audio api. im not sure who (or what) did it first, but av processing tends to converge on the idea of "nodes" that produce, transmute, or consume samples. i'm pretty sure this comes from the status quo before large investments in pro-av workflows on computers, which i'd guess would be modular synths.

the hum

i didn't do much analysis on this asset in particular. i assumed (and was correct) that the loop is mostly a perfect loop. a friend of mine helped get it sample correct so that it'd loop without a click when playing with web audio api (you can instruct a "sink" node to continue supplying samples by resetting it's offset), but there wasn't any other post processing on that asset. the audio node graph is pretty simple at this point: source node (hum.wav) -> gain node (-25db) -> output node. the gain node is there to match the volume adjustment made in game. the interesting (but only relatively) part of the graph however is the whir nodes.

the whirs

ffxiv plays the whirs at random intervals, pitches, and gain (volume). these are all added to create an illusion of life(?) that make it harder to detect when an asset is repeating. thankfully, ffxiv also provided values that gave the min and max values for each of the random parameters, so it was easy to translate into javascript:

// select a random whir
const whirIndex = Math.floor(Math.random() * whirs.length);
// select a random pitch
const whirPlaybackRate = Math.random() * (1 - 0.794) + 0.794;
// select a random volume
whirGainNode.gain.value = Math.random() * (1 - 0.6) + 0.6;

if you've ever built a graph by hand the flow is usually something like: allocate a node, set metadata, and connect. i do the following to setup the hum:

const humSource = audioContext.createBufferSource();
humSource.buffer = await loadSample(
    audioContext,
    isSafari ? "assets/hum.wav.opus.aac" : "assets/hum.wav.opus",
);
humSource.playbackRate.value = 0.63;
humSource.connect(humGainNode);
humSource.loop = true;
humSource.start(0);

it's actually ok that i connect the node before i set loop, because the node doesn't produce samples until i call start.

the other part of the ceremony is the whir loop. it's not actually a loop using conventional loop control flow. instead of using a while (true) with a random "sleep" in between, i instead use setTimeout to schedule the next whir at a random interval.

function chooseWhir() {
    const whirSource = audioContext.createBufferSource();
    const whirIndex = Math.floor(Math.random() * whirs.length);
    const whirPlaybackRate = Math.random() * (1 - 0.794) + 0.794;
    whirGainNode.gain.value = Math.random() * (1 - 0.6) + 0.6;
    whirSource.buffer = whirs[whirIndex];
    whirSource.playbackRate.value = whirPlaybackRate;
    whirSource.connect(whirGainNode);
    whirSource.start(0);

    whirSource.onended = () => {
        // disconnect self and re-queue another whir
        whirSource.disconnect(whirGainNode);
        const nextWhirDelay = Math.floor(Math.random() * 2001);
        setTimeout(chooseWhir, nextWhirDelay);
    };
}

same deal here: create a source node, select a random asset, playback rate (pitch), gain (volume), and connect it to the gain node (which stays constant in this process and only has it's value changed.) the key here is that when the source asset ends, instead of looping, we remove that node from the graph and add a new one (by calling chooseWhir again.) because there is an expected delay before the next whir, i'm ok with adding whatever latency is added by doing the random calculation, it's likely marginal.

conclusion

this was a fun project to work on. you can demo the final result here. because we've ruined playing audio in browsers you need to interact with the page first before you can hear anything, so make sure you unmute both the whir and the hum and click "connect" to listen.

vanilla js

growing up as an engineer i didn't care too much about the web or web technologies. i was more focused on systems applications, games, and networking. im not sure where i learned to interact with the web or build web apps, but it sure wasn't the bog standard npm + react + jsx/tsx dance we have today. im not building enterprise grade web apps meant for collaboration between 100+ devs, it's just me. i didn't mind creating static elements and using document.getElemenyById. it greatly simplified the creation process and let me move quicker and care about the project at hand. no frustrations with transpilation or jsx, my project wasn't that complicated.

Emacs 31.0.50 (Org mode 9.7.9)