nesbox / TIC-80

TIC-80 is a fantasy computer for making, playing and sharing tiny games.
https://tic80.com
MIT License
4.91k stars 474 forks source link

[request] import sound tool/cmd (wave data not tracker) #2162

Open atesin opened 1 year ago

atesin commented 1 year ago

hi... the documentation is very swallow... but i was reading the "import" console command, not too much explanatory

as far as i understood, we can open tic-80 folder ("folder" command) and once opened we can place any file we want, for each tic80 "editor":

but what about SOUND? ... is very tedious having to plot waves dot by dot, moreover i cant imagine how it will end sounding, is more like a trial-error process... it is also very hard to try to match some sound with other "real" one, would be great if we could just sample, lower the quality and just import

would be great if import could also work with sound files (wav?), and the documentation specifies the process and wave format (frequency, bit resolution, which voice/channel, etc)

.... in case it doesnt support.. if does could you please update documentation in details?... thanks

atesin commented 1 year ago

thank you....

one last question.... which frequency (or note) the imported .wav file should have to fit in a tic80 waveform slot?? .... because as i understood, that 32 samples means a single cycle, then that soundwave will be stretched across time according the note is being played

so if i save the .wav file lets say at 150hz and import to tic80 soundform, but tic80 waveforms are 300hz (as examples), then i will be importing just half cycle... same the opposite, if i save a 600hz .wav and import to tic80 then my sample will sound too pitched and i will lose resolution

or like somebody said above, that a single cicle lasts 1 tick (1/60 sec), in which case a single waveform will have 60hz ?? ... or depends on other conditions?

bztsrc commented 1 year ago

Neither, these are much higher level concepts, and as I've said, wav2tic does not do any of that: no Hz conversion, nothing fancy. It cares about the format representation only. It just takes 32 samples from the .wav, and saves as TIC-80 and that's it. That's why I wrote you should prepare the wave beforehand in Audacity to have exactly 32 samples (which could be 2 secs with 16Hz or half a sec if it's 60Hz, doesn't matter, only the number of samples matters).

This PoC proves that TIC-80 could be able to import .wav files; one can add these high-level resampling / retuning etc. filters later, when the .wav import is already working (independently to the format representation). FYI don't expect good results from these filters, not much point in converting sophisticated wave samples into chiptunes (with only 16 different possible values in each 32 samples).

Cheers, bzt

atesin commented 1 year ago

hi @bztsrc ... maybe i didnt explain myself enough

i understand wat2tic doesn't make any sound processing nor conversion (other than signed/unsigned), but just read the .wav file, take it here and put in there

i meant... when i am sampling the sound, with audacity or similar, and i select a slice that i think it sounds clear, and before save the .wav file i should stretch the sound frequency (resample?) to adapt tic80 to fit in his wafeform slot.... as part of the work we do in our sound editor gui/cli program

so... which is the frequency we should pitch our samples in our sound editor program before saving to .wav, to make them coincide tic80 waveform slot ... is it 60hz?... imagine this bad example:

image

bztsrc commented 1 year ago

Here's a slightly modified version. For convenience I've swapped the signedness to match Audacity's, so this assumes unsigned 8-bit data by default, and requires the -s flag for signed samples. wav2tic.zip

which is the frequency we should pitch our samples in our sound editor program before saving to .wav, to make them coincide tic80 waveform slot ... is it 60hz?

This should be obvious. Frequency is the number of samples per second, so it depends on the length of your wave. For example, if you have 2 secs of wave, then frequency has to be set to 16 Hz to get 32 samples. If your wave is 1 sec in length, then your freq would be 32 Hz to get 32 samples.

But you don't have to do this calculation yourself, Audacity is perfectly capable of working with the number of samples. Just click on "s" after the time, and select "samples".

imagine this bad example

I believe you still don't understand what chiptune is. As @joshgoebel said, 32 samples isn't much. It is actually a very very extremely low number of samples, only capable of doing chirps and beeps. Do not expect a dog's woof with such a low resolution (32 samples of -8..7), that's never going to happen.

Try something like this instead: wav2tic_example

Cheers, bzt

atesin commented 1 year ago

so the answer is 32hz ??? or 1hz ?? ... 440hz?... 1378.125hz??

image

i understand that with common sampling rates, 32 samples is very short, just about a click sound, less than a click, the click of a click, actually a single cycle of 1000hz have JUST 1ms!, barely audible .... i believe tic80 plays this single cycle in loop, at different speeds according note, that is because it seem to be continuous and larger.... if we vary some dots in waveform it seem to change harmonics and will sound like another different thing

so, if the sound has for example 44100hz resolution, each sample is taken each 1/44100sec = about 0.000023 secs! .... yes but i have to stretch and resample that sound to make it fit with tic80 waveform single cycle, and export just a single cycle that fits within there..... another info that could be useful is tic80 sound sampling rate in hz that is not mentioned anywhere, to do calculations.... in other words

i think... in audacity at least, with sampling rate of 44100hz, to make a single cycle have 32 samples, i have to stretch+resample any sound to 44100/32 = 1378.125hz... that way if i zoom the sound enough it look like cycles have exactly 32 samples:

image

generated sine wave of 1378.125hz, zoomed to 1 cycle

i can't make it work.. it stills looks a single high dot in tic80 waveform... maybe are the id3 tags? but i dont add any... i also tried exporting with nyquist wav 8bit signed format but the same

so back to original question.... which freq to STRETCH+RESAMPLE sound to make 1 cycle have 32 samples??, 32hz ??? or 1hz ?? ... 440hz?... 1378.125hz?? ... some formula??.. it depends on something else?

p.s.: about -u .... i did learn a little more about exporting thanks to you, but i think the original wav2tic sign handling is ok because it uses defacto standard and rarely audacity doesn't ... a better improvement could be to supply an output file name to "inject" wavs, but may be a little harder, perhaps in some future :) (be patient, atesin!)

bztsrc commented 1 year ago

i believe tic80 plays this single cycle in loop, at different speeds according note, that is because it seem to be continuous and larger

Yes, exactly. That's why there's no point in talking about the frequency of one wavepattern. It only has a frequency when actually being played as a note.

another info that could be useful is tic80 sound sampling rate in hz that is not mentioned anywhere

That's because there's no such thing. You have 32 samples, which are repeatedly taken. How often the pointer jumps to the next sample in the wavepattern depends on the pitch, TEMPO and SPEED values when it's being played. Of that I'm certain, but how exactly pitch, TEMPO and SPEED is used to calculate the duration of a single sample (which would be needed to tell the Hz) is a mystery to me, and sadly @nesbox did not answer my question here nor your question here. So we can only guess...

that way if i zoom the sound enough it look like cycles have exactly 32 samples

Right, but your problem is, it shouldn't be just one cycle when you zoom in, rather the entire wave altogether can't be longer than 32 samples. That's why you're getting bad results.

In other words, the .wav file must be 76 bytes long, no more (assuming no meta info). To give a fully technical and proper explanation, if you take a look at the .wav with a hex viewer, you should see that the "data" chunk's length is 0x20, aka. 32 bytes exactly (for mono 8-bit samples), regardless to the frequency. wavdump

Hope this helps, bzt

atesin commented 1 year ago

wavepattern depends on the pitch, TEMPO and SPEED values when it's being played.

you are right

that way if i zoom the sound enough it look like cycles have exactly 32 samples

Right, but your problem is, it shouldn't be just one cycle when you zoom in, rather the entire wave altogether can't be longer than 32 samples. That's why you're getting bad results.

actually the test i did is to generate a mono sine wave of 1378.125hz 1sec duration... then zommed, then select a single cycle and noted it has exactly 32 samples... then i copied the selection (1 cycle, 32 samples) and PASTED to a new project... then export the 1 cycle 32 samples project to .wav 8bit, not the 1sec lenght one

i understand there is no way i could fit 1sec sound in a 32 samples slot .. for god sake, don't make me repeat this again :(

which i want to know is to which frequency i should stretch+resample my sound (whatever frequency and lenght it has), in order to make each cycle have exactly 32 samples... so that exporting a single cycle would fit exactly in tic80 waveform slot window .... i dont care the later playing speed

for example...

(extract, actual sound is larger) image

zoomed in to view 1 cycle image

need to be resampled before exporting... to make the same sound more pitched, enough to make cycles fit in tic80 wave slots .... so that for example, if i play that sound in tic80 at G-3 (~200hz), it sounds "similar" to original sample (or at least recognizable.. i know it will sound more distorted)

..... so... to what frequency we have to stretch samples before exporting, to make it tic80 wave slot holds exacly 1 cycle of that sound?? (neverthless of playing speed)

p.s.: i can't make wav2tic work properly... maybe the windows version has some issue/dependency

bztsrc commented 1 year ago

i want to know is to which frequency i should stretch+resample my sound (whatever frequency and lenght it has), in order to make each cycle have exactly 32 samples

It's not "make each cycle 32 samples". You must fulfill the equation freq*total wave length=32 to get 32 samples in total.

so that exporting a single cycle would fit exactly in tic80 waveform slot window

I've achieved this by using 32Hz and making the entire wave 1 sec long. But other combinations will work too, it is just 32Hz/1sec is the simplest to work with.

Is this extremely short? Yes, you're right it is. And even its precision is limited to the -8..7 range. This can really just store some rudimentary chirp or beep.

to what frequency we have to stretch samples before exporting, to make it tic80 wave slot holds exacly 1 cycle of that sound?

If you were asking about MEG-4, then I could properly answer you question: the output audio freq is set to 44100Hz, feeded at 50Hz, one wavepattern can be 16376 samples long each in the range -128..127 tuned at 262Hz pitch, and for playing that wave at grand piano's middle C it needs 882 samples per tick (because 44100/50*262/262=882). (Eh, 428 is the period of C-4, it's freq is 262Hz)

No clue how this is calculated in TIC-80 though, sorry. :-( The TIC-80 wiki isn't helpful, it only says this calculation is done, but does not say how exactly: The frequency value is converted to a period value (clamped to 10...4096) suited for a virtual 2,088,960 Hz clock. This period value produces an update frequency that is close to 32*frequency (for waveform envelopes) or 16*frequency (for noise output). Total gibberish, right? Period value of what? And without knowing how long one sample lasts, it is impossible to tell what the output frequency actually is (is it 32*2,088,960/4096=16kHz tops?), so I don't know how to resample/stretch your wave in Audacity so that it will be reproduced at the same freq in TIC-80, sorry. Use trial-and-error, that's the best advice I can give you.

i can't make wav2tic work properly... maybe the windows version has some issue/dependency

In the first version I forgot to account for the chunk's length that's also stored in the data chunk, so it reads 4 bytes shorter wave (with mono 8-bit that's 4 samples short, with stereo 16-bit that's 1 sample short). Anyway, this has been fixed in the last version. Neither the Linux, nor the Windows version has dependencies whatsoever, btw.

Cheers, bzt

bztsrc commented 1 year ago

i want to know is to which frequency i should stretch+resample my sound (whatever frequency and lenght it has), in order to make each cycle have exactly 32 samples

I'll try to explain why your question doesn't make sense, but I must stress I'm not sure about the TIC-80 part.

Let's assume we have fixed frequency (let's say 44100Hz), that means 44100 samples are played in each second.

       one cycle
 /------------------\
 ###                 ###                 ###
 ###                 ###                 ###
+---++---++---++---++---++---++---++---++---++---+   (10 samples)
---------------------------------------------------> time

To change the pitch, you make the peeks in the samples more dense (or you use more gaps between them), like this:

    one cycle
 /-------------\
 ###            ###            ###            ###
 ###            ###            ###            ###
+---++---++---++---++---++---++---++---++---++---+
---------------------------------------------------> time

Note that the number of samples changed within one cycle, however the overall number of samples in a second hasn't, that latter is still 44100 samples per second. Therefore, no matter the pitch, one sample is played for 1/44100 seconds long.

This is how Audacity handles things and this is what you're asking about when you talk of cycles.

However, if I'm correct this isn't how TIC-80 works at all. As far as I can tell, it instead messes with the update frequency, like this:

   one cycle
 /----------\
 #           #           #           #
 #           #           #           #
+-++-++-++-++-++-++-++-++-++-++-++-++-+
---------------------------------------------------> time

Now the number of samples in a cycle hasn't changed, but the number of samples in a second did. Despite of the same wavelength, the peeks in the wave got closer in time, so the pitch indeed changed in this case too.

To get the wave's frequency you should not ask how many samples are there in a cycle, rather you'll need to know for how long one sample is played for a certain pitch (a question that the wiki does no answer well, so I dunno :-( ).

But I'm no expert on TIC-80 sounds, so we really need @nesbox to confirm this. Until then, trial-and-error is you best shot to get the same pitch in both Audacity and in TIC-80.

Cheers, bzt

atesin commented 1 year ago

yes i know by stretching/resampling sounds a kind of interpolation is made... sampling rate is not changed... like shown in this example:

image

let me express my doubts in another way...

in other words.... imagine i have a repetitive sound like me saying "aaaaaa", in a common .wav or .mp3 file (*) .. what processing should i apply to sound, to finally make tic80 play it and be (kind of) recognizable??

... in other words... how can i make tic80 play "aaaaaa" with my voice (more or less), if i had recorded previously?

.... or if i have, let's say, the recording of a clarinet.... what should i do to copy that sound (a single note) to tic80 and make it sound aprox. like that clarinet?

.... all of these suposedingly without shifting notes in tic80... i mean if we play a C-5 it sounds with the frequency of a C-5, while making sound still rather identifiable

(*) common format: like 44100hz sampling rate, 32bit resolution, stereo, 128kbps or more

bztsrc commented 1 year ago

sampling rate is not changed

Yes it is changed, on the TIC-80 side. I'm certain of that, I just don't know how much it changes for a certain pitch.

what processing should i apply to sound, to finally make tic80 play it and be (kind of) recognizable??

No such processing exists.

what should i do to copy that sound (a single note) to tic80 and make it sound aprox. like that clarinet?

I've said this multiple times, don't expect any recognizable sounds, no matter what filter/resampling/whatever magic you use. No dog's woof, no "aaaaa", no clarinet, no nothing. Just chirps and beeps, that's the best 32 samples in the range -8..7 will give you.

Cheers, bzt

atesin commented 1 year ago

sampling rate is not changed

Yes it is changed, on the TIC-80 side. I'm certain of that, I just don't know how much it changes for a certain pitch.

right... but i meant in a sound editor program... when you change the tone or apply some effect to some sound it does by interpolation not by changing sampling what keeps constant everytime (unless you change through menu/tool).... but for tic80 i am sure is easier and cheaper to achieve tones by changing sampling rate instead doing interpolation

what processing should i apply to sound, to finally make tic80 play it and be (kind of) recognizable??

No such processing exists.

if you read a little more carefully you will realize i used sentences like "kind of", "more or less", "aprox.", "still rather", etc ... I AM TOTALLY AWARE IT WON'T SOUND PERFECT ...

i feel you treat me as if i would fool or ignorant .... i worked on ataris when i was in the school making programs in basic, and worked as dj when in college, so i know something about microcomputers and music

it can't be, there must be some way(s) to do it, with the help of your tool... even you converted a sine sound to be exported to wav, to be imported to tic80, and worked... what i am asking is not to only convert 1hz 32samples generated waves, but also any other sampled sound whatever it ends sounding in tic80... we have all the tools

try to be more imaginative

bztsrc commented 1 year ago

what i am asking is not to only convert 1hz 32samples generated waves

The tool does not only work with generated waves, you can supply whatever data you'd like. But the wave has to be 32 samples long in total, because there's no more storage space in the TIC-80's memory. There's nothing you can do about that limitation (unless you rewrite TIC-80 itself to support more).

try to be more imaginative

I am. That's why MEG-4 SFX stores waves up to 16376 samples with the resolution of -128..127. That's still a chiptune and distorted like hell, but at least that's enough information to do what you're asking for. (Check out the demo, around 1min there's a man shouting "come on". Not perfect by any means, but clearly recognizable.)

32 samples with the resolution of -8..7 loses so much of the wave data that it's simply not enough information to be recognizable. That's why people call it a chiptune and not audio.

Cheers, bzt