davircarvalho / Individualized_HRTF_Synthesis

Synthesis of individualized HRTFs based on Neural Networks, Principal Component Analysis and anthropometry
GNU General Public License v3.0
62 stars 7 forks source link

Synthesis of Near field HRTF and Denser grid #6

Closed ABoredBunny closed 1 year ago

ABoredBunny commented 1 year ago

I first of all want to thank you for making this, this is the most convenient and user friendly way of making a personalized HRTF yet. Going from an anechoic chamber with speakers. To needing a dense 3d model of ones head and a decent computer. To only needing a caliper and about a minute of compute time, has been quite the trip.

But this solution still has problems: This solution only outputs one field it doesn't produce fields that can be used for near field synthesis. And it only outputs a set field with specific density.

For the first one, I suggest this paper here's someones implementation in matlab. This approximation creates pretty convincing results, it is implented in Binaurilizer_nf. Something like a field where you can type in distance and generate a field, or specify multiple fields during generation.

The second one is up to how you may implement it, what I would want is the ability to interpolate the points to an arbritrary grid. Possibly using a config file which specifies the number of rows and there spacing in degrees or similar. Because some things don't have there own interpolation, Or expect certain grid types namely OpenAL Soft.

Also, somewhat unrelated, but you should probably mention that these HRTF files are meant to be used together with Hesuvi HP filters. Because they sound wrong with other harman target equalization because they EQ after 10KHz.

davircarvalho commented 1 year ago

Hello @ABoredBunny, thanks for your interest in this repository and your feedback! I'll try to answer your questions and clarify some points as I can.

This solution only outputs one field it doesn't produce fields that can be used for near field synthesis.
And it only outputs a set field with specific density.

I will assume what you are referring by "field" concerns the measurement positions or the sampling grid (if it's not it, please forgive me) -- There's a very specific and boring reason why this sampling grid was chosen, and it's related to the datasets used to train this model (there's a whole paper about this under peer-review, so I can't share it with you right now). But what really matters is that this repository already contains an interpolation function you can use to generate HRTFs for whatever direction you want. This function has at least 4 interpolation methods to explore, go nuts. But keep in mind the output grid from the app will not work well with spherical harmonics interpolation or extrapolation, and bilinear interpolation will not work for positions beyond the the elevation limits of this grid, other than that, this function will let you output the relevant grid for most use-cases.

About near-field extrapolation, I've invested a lot of time doing it for the auralization app . But honestly, I have no motivation to bring that to the individualization app. Although if you have HRTFs that cover the full sphere (not the ones from the app) you could simply use the other function I mentioned above, for interpolation, set the flag for spherical harmonics and provide the azimuths, elevations and distances you desire (just check if it's using the ITA api, it's the only one that includes distance with spherical harmonics). AFAIK that should still be state-of-the-art. If you need help generating a grid, a good starting point is this function.

but you should probably mention that these HRTF files are meant to be used together with Hesuvi HP filters. Because they sound wrong with other harman target equalization because they EQ after 10KHz.

Sorry, I honestly don't know what you mean by that, HRTF spectra above 10kHz contain the monoaural cues for localization related to the pinna shape. Hesuvi HP filters, I assume are headphone correction filters, they are supposed to reduce the influence of the headphone itself in the auralization process, that's a totally independent procedure, that has a separate physical meaning from HRTFs. Harman target equalization is the only EQ per se, as it equalizes the content to have a balance that "most people like". So ... I'm not sure what's your point :)

ABoredBunny commented 1 year ago

Thanks for the quick reply :) The thing about HP filter was, that I wasn't sure which Equalization filter is the correct one. Because the Hesuvi filter are completly flat around 10k(at least on my headphones) while others like the oratory1990, EQ that area as well, and I wasn't sure what EQ filter the HRTFs are generated for, considering that the harman target is partialy based on estimations of ear and head. And I wasn't sure how this accounts for it.

I'm sorry to say this, I have no idea how to use matlabs, I don't mean rewritting the script so it outputs what I want. I mean once I rewrote it, how do I make it process stuff? Do I just drag it in there?

By the way, It would be helpful if it was added in an advanced tab in the app, if the function is already there.

Edit: Also, does the distance stuff include the near field bass effect?

davircarvalho commented 1 year ago

Hello,

The latest commit includes custom equiangular grid output, I hope that's what you're looking for. I will soon update the standalone version as well.

About the near field, the model I have in the other app includes the effect of level difference change with frequency or as you put it "near field bass effect", this part was insppired in the 3dtunein app from imperial college, they have a paper about it if you wanna know how it works. If you're curious have a listen to it and let me know what you think, here's the link.

As I mentioned before integrating that code here is quite a big effort like a week of work, but I only work on this on my free time. But if you think it is something more people would be interested I could create a goal in the github sponsors and make it happen

ABoredBunny commented 1 year ago

I will be honest, as much as I would want it to be a feature, it is very niche in something that is already niche. And most people that are aware of it probably are capable of using Mesh2HRTF. I mostly thought of it for an average enduser, because your standalone app makes personal HRTF accesible for most people.

If it takes a whole week, then you probably shouldn't.

I thought more of a simpler implentation, one where you apply EQ filter based on something like a lookup table.

Something that works fine to add a bit of depth to the soundscape. It wouldn't have to be ITD, Parallex corrected "Academicly correct". Something that works fine for video games.

Also, thanks for adding the equiangular grid output. It should allow for Regular use in OpenAL soft, The delay issue has already been fixed.