creating the aetheryte radio
sometimes when i work i like to listen to ambiance. one video in particular is the "aetheryte asmr", which is an in game recording of the aetheryte crystal. for the past 6 years i've been listening to the same youtube video on repeat. there are a number of problems with this approach:
- the youtube player's "loop" functionality doesn't always work. it might loop 20 or 200 times before failing(?) and auto-playing another video. this is jarring.
- the audio in the video is a ~7 minute recording, but the loop isn't seamless (i spent far too long re-architecting my music player's internals to support gapless playback), and while it's long enough that i don't care when it cuts and loops, it would be nice if it were seamless.
- of course i can't keep the game open.. that's far too resource intensive, other players (ffxiv is an mmorpg) could approach nearby and create noise and ruin the effect, etc.
none of these were deal breakers for me. 6 years is a pretty long time to go facing some struggles before sitting down and deciding to tackle a problem.
nevertheless, one day i finally sat down and thought to myself.. "how can i get the best experience for the ambience without sacrifice?"
the first answer is that i needed to get the game assets. whatever the game was using to play the noise, i needed to extract. with those in hand, i could recreate or play with them in whichever way i wanted.
getting the source assets
ffxiv isn't a super well protected game (at least i don't think.) if you web search "ffxiv data explorer" there are old java apps that load up the asset packages from the games install directory and let you export (presumably) compressed assets. i don't remember which fork i used, but i do know that it came with a hash table that mapped the file to an english file name.
you're going to laugh at me (and honestly that's ok.) i spent 4 hours coming through 4000 audio assets trying to find the audio assets. i literally ctrl+a'd all 4000 .wav files and pressed the left and right arrow keys to move back and forth within the large mpv playlist, looking for the aetheryte noise. the funny part is that i didn't find them, and had to go to sleep at 3am.
i'm an engineer though, right? work smarter, not harder (why didn't i do this before doing the tedious work? idk.) was there any better way to get the sound? did i miss it during my 3am comb over? (hint. i did.) turns out, there is! ffxiv has an extensive modding community and platform, and while i don't play the game much anymore, it was exhilarating to see how much effort people put into customizing their experience.
xivlauncher, vfxeditor, and soundfilter
if you boot ffxiv using xivlauncher (endorsed by my roommate who has
180 days of play time) you can load up plugins. one plugin of interest was
soundfilter. i know the name is a bit misleading, but it has the
ability to display the names of now playing assets. this helped a
ton because i was able to teleport to a new area in the game (the
aetherytes are basically waypoint / teleport crystals) and figure out
which sound was being played. quickly after i found that the asset i
was looking for was
bgcommon/sound/fst/placednpc_ethelight_big_loop.scd/0
. i quickly
exported said asset and was on my way.
an interesting note about that asset is the /0
at the end. i'm
pretty sure the sound "asset" contains multiple sounds (i think they
were called tracks in game.) 0 was simply the "hum" or the low
frequency of ambiance.
sadly, getting the actual ambiance would require a bit more elbow grease. i could have just thrown these together in logic pro and called it a day, but where's the fun in that? plus, i'd have to create an excessively long track. i didn't want to be subconsciously aware of when the loop repeats. i wasn't too worried about this though, as i had the right tools up my sleeve to solve my first world made up problem.
web audio api
enter the web audio api. im not sure who (or what) did it first, but av processing tends to converge on the idea of "nodes" that produce, transmute, or consume samples. i'm pretty sure this comes from the status quo before large investments in pro-av workflows on computers, which i'd guess would be modular synths.
the hum
i didn't do much analysis on this asset in particular. i assumed (and
was correct) that the loop is mostly a perfect loop. a friend of
mine helped get it sample correct so that it'd loop without a click
when playing with web audio api (you can instruct a "sink" node to
continue supplying samples by resetting it's offset), but there wasn't
any other post processing on that asset. the audio node graph is
pretty simple at this point: source node (hum.wav) -> gain node
(-25db) -> output node
. the gain node is there to match the volume
adjustment made in game. the interesting (but only relatively) part of
the graph however is the whir nodes.
the whirs
ffxiv plays the whirs at random intervals, pitches, and gain (volume). these are all added to create an illusion of life(?) that make it harder to detect when an asset is repeating. thankfully, ffxiv also provided values that gave the min and max values for each of the random parameters, so it was easy to translate into javascript:
// select a random whir const whirIndex = Math.floor(Math.random() * whirs.length); // select a random pitch const whirPlaybackRate = Math.random() * (1 - 0.794) + 0.794; // select a random volume whirGainNode.gain.value = Math.random() * (1 - 0.6) + 0.6;
if you've ever built a graph by hand the flow is usually something like: allocate a node, set metadata, and connect. i do the following to setup the hum:
const humSource = audioContext.createBufferSource(); humSource.buffer = await loadSample( audioContext, isSafari ? "assets/hum.wav.opus.aac" : "assets/hum.wav.opus", ); humSource.playbackRate.value = 0.63; humSource.connect(humGainNode); humSource.loop = true; humSource.start(0);
it's actually ok that i connect the node before i set loop, because
the node doesn't produce samples until i call start
.
the other part of the ceremony is the whir loop. it's not actually a
loop using conventional loop control flow. instead of using a while
(true)
with a random "sleep" in between, i instead use setTimeout
to
schedule the next whir at a random interval.
function chooseWhir() { const whirSource = audioContext.createBufferSource(); const whirIndex = Math.floor(Math.random() * whirs.length); const whirPlaybackRate = Math.random() * (1 - 0.794) + 0.794; whirGainNode.gain.value = Math.random() * (1 - 0.6) + 0.6; whirSource.buffer = whirs[whirIndex]; whirSource.playbackRate.value = whirPlaybackRate; whirSource.connect(whirGainNode); whirSource.start(0); whirSource.onended = () => { // disconnect self and re-queue another whir whirSource.disconnect(whirGainNode); const nextWhirDelay = Math.floor(Math.random() * 2001); setTimeout(chooseWhir, nextWhirDelay); }; }
same deal here: create a source node, select a random asset, playback
rate (pitch), gain (volume), and connect it to the gain node (which
stays constant in this process and only has it's value changed.) the
key here is that when the source asset ends, instead of looping, we
remove that node from the graph and add a new one (by calling
chooseWhir
again.) because there is an expected delay before the
next whir, i'm ok with adding whatever latency is added by doing the
random calculation, it's likely marginal.
conclusion
this was a fun project to work on. you can demo the final result here. because we've ruined playing audio in browsers you need to interact with the page first before you can hear anything, so make sure you unmute both the whir and the hum and click "connect" to listen.
vanilla js
growing up as an engineer i didn't care too much about the web or web
technologies. i was more focused on systems applications, games, and
networking. im not sure where i learned to interact with the web or
build web apps, but it sure wasn't the bog standard npm + react +
jsx/tsx dance we have today. im not building enterprise grade web apps
meant for collaboration between 100+ devs, it's just me. i didn't mind
creating static elements and using document.getElemenyById
. it
greatly simplified the creation process and let me move quicker and
care about the project at hand. no frustrations with transpilation or
jsx, my project wasn't that complicated.