Audio sample

[RecursiveCall]

A sample on how to use Audio (Effect and/or Music) with the Toolkit is a major missing piece of the puzzle.

[ArtiomCiumac]

Toolkit is focused mostly on graphics (+ a little bit on input) and does not have specific support for audio - it should be used exactly in the same way as for any other engine.

[xoofx]

@RecursiveCall, It is not only a sample but a full assembly/API (SharpDX.Toolkit.Audio) that needs to be developed to support it on all platforms and provide an abstraction of MediaEngine/XAudio2. I agree that this is a missing feature. A basic API that supports music and effect is not that difficult to develop with the underlying API, a couple of hours (a bit more to support from Windows 7)

I have currently no time left to start developing this, not sure about @ArtiomCiumac?

But @RecursiveCall if you have some time to boostrap this part (using the GameSystem infrastructure like how it was done for Input), we could integrate it in a branch and push it to the main branch as soon it is feature completed (note that by bootstrap, the code should still be clean&commented code ;) )

[ArtiomCiumac]

not sure about @ArtiomCiumac?

I think I can find some time, most probably I will take a look on what we can do this weekend.

[RecursiveCall]

@xoofx While I do have a bit of bandwidth, I am still less familiar with codebase then you and @ArtiomCiumac. On top of that, my previous attempts for content extension and particle samples were not quite "publish-worthy".

If @ArtiomCiumac can at least put something together in the Toolkit level, I can try my best to create a good Toolkit sample.

[xoofx]

DirectXTk has recently added support for Audio (at least samples):

[xanather]

Audio support is the last major feature that is needed for the Toolkit (in my eyes at least). I have no XAudio2/low level programming knowledge otherwise I would implement it myself.

Just a SoundEffect/Song/SoundEffectInstance and MediaPlayer should be enough for many scenarios. For now I may try using another .net high level audio library for sound playback.

[dfkeenan]

@xoofx & @ArtiomCiumac, I have started "bootstrapping" something for this. Work in progress in a branch of mine: https://github.com/dfkeenan/SharpDX/tree/Toolkit_Audio . It's functional in that you can load a SoundEffect via the ContentManager and play it. It has basic instance pooling. You can change volume, pitch and pan. Still missing 3D audio, and plenty of document comments (and probably other stuff).

I have also added a Desktop demo. I have only tested this code with .Net 4.0, not sure if there are any special cases need for other platforms.

I might not get back to it till after Christmas now. If you want me to request a pull so you can put it in a branch or something let me know.

[dfkeenan]

I have now ported WaveBank from DirectXTk as well. I have tested it with a couple of files seems ok except for the Xbox files. Xbox uses big endian, the reader reads most of the file correctly except for the actual audio stream. The reader from the dxtk does not seem to specifically change the endianness of the audio stream so I am not exactly sure what to do here. My guess is you would have to take sample bit rate into consideration when changing the endianness.

I did not port all the Xbox(one) specific code like xMA format, Leaving the Xbox format as unsupported is also an option.

I have not tested a large selection of formats or wave bank files.

[RecursiveCall]

@dfkeenan I'll be testing your stuff on the weekend and will provide feedback.

[xoofx]

@dfkeenan thanks a lot for bootstrapping this. I will try to have a look soon and provide you as well some feedback. Also don't hesitate to go outside of beaten paths in terms of API conformance with XNA when It can bring some enhancements. Also try to add comments description to the public API whenever you consider it stable.

[dfkeenan]

FYI I just fixed compatibility issues with .NET 2.0 it now builds under that as well.

@RecursiveCall cool let us know if you have problems / API ideas.

@xoofx,

No worries, I did it out of boredom/interest. I don't have any immediate use for it myself, I just thought it would be a worthwhile contribution. I didn't have any prior XAudio knowledge (or much DirectX for that matter), I just hope I don't loose interest before it's "done".

Feedback always welcome. I "copied" the XNA API mostly from laziness/lack of any creative ideas, if yourself or others have some good ideas for a better design I am happy to give them a try.

After messing with this WaveBank stuff for a few days I am not so sure if it's that useful of a file format. Personally I would probably prefer to load a "Wave Bank" from say a yaml file and a bunch of loose audio files. So at this point not sure that I will also port the XWBTool as well unless there is demand for it.

Other things I would like to add:

Support for encoded audio formats (mp3 etc). I could possibly use the AudioDecoder class (saw it in one of the samples) to get the pcm stream and put it into a SoundEffect. Though having separate classes for music might be a better idea.

I did plan on putting document comments on the public API once I thought it was stable from a design/bug perspective.

Anyways this might be the last you here from me until the new year. Happy holidays peoples.

cheers, dfkeenan

[RecursiveCall]

@dfkeenan I tested the desktop project. The audio played perfectly. However, I tried dropping the class and content into a WinRT project and got nothing but compilation errors. I rebuilt SharpDX with the AudioManager for Win8Debug, and while the assembly was added successfully and the namespaces show up in the object browser, for some weird reason VS complains about not being able to reference any of that. Tried VS2012 and VS2013.

Did you try WinRT/WP8? If you did/do and you run into the same issue, I'll investigate. Otherwise, any ideas on what went wrong in my workspace? (I'm on Win8.1)

[dfkeenan]

@RecursiveCall Yeah I have been having the same problems. I could never get the Win8Debug builds to work before I added the project, I don't think I installed the Windows Store App stuff when I installed VS2012. I upgraded to VS2013 on Friday (ticked all the boxes this time :D) so now the code gen "works" on build, but the audio stuff I added does not (can't find SharpDX.IO or SharpDX.XAudio2).

I have no idea what I have done wrong. I am currently running VS2013 Pro on Win 8.1 (I uninstalled VS2012) and Have the June 2010 DirectX SDK installed (not sure if you need something extra/else for Win8 build). Maybe if @xoofx gets a chance to look he might be able to see what I have screwed up. But in the meantime I would be grateful if you could have a look and see if you can locate the problem. Chances are I won't get a good look at this again till the new year.

[RecursiveCall]

@dfkeenan Well, at least it's not just me. I'll continue investigating over the holidays.

[dfkeenan]

@RecursiveCall I have just fixed it so Win8 builds, i have not tested that it works on that platform yet.

[dfkeenan]

Update:

• It's all loopy (added SoundEffectInstance.IsLooped).
• That's spatially good (added SoundEffectInstance.Apply3D and SoundEffectInstance.Apply2D). I have not fully tested these, the demo only tests Apply3D. There were a few things I am not sure about, I have put some to-dos in there as reminders. Also I did these a bit different from how Xna etc did it.....

public void Apply3D(Matrix listener, Vector3 listenerVelocity, Matrix emitter, Vector3 emitterVelocity)

public void Apply2D(Vector2 listener, Vector2 listenerVelocity, Vector2 emitter, Vector2 emitterVelocity)

• Vrrm Vrmm, suped-up the demo, it now does more stuff and is a bit more user friendly.

I am still trying to make some design decisions, mainly around when to throw exceptions and when to just handle it i.e. should the Play method throw an exception if it can't be played or return a Boolean. Also should a property throw exceptions if value is out of range or just clamp them. I have still barely scratched the surface in regards to document comments.

I am still considering adding something along the lines of a SoundTrack/Music class for files like mp3s with the ability to fade-in\out. If I did add this I would also like to add a SoundTrackVolume and SoundEffectVolume to the AudioManager so the volumes could be controlled separately, still no idea how to accomplish this. I should probably tidy-up/comment whats done first, it would be good to get v1 complete before the next release I suppose.

[RecursiveCall]

@dfkeenan An update from my end. I managed to get your code going in WinRT. All four options work as with the desktop build. The app does not pass the certification tool, but after testing some of the other SharpDX samples, it appears to be the case with the default SharpDX sample projects. @xoofx and @ArtiomCiumac, the specific failures are:

◦API D2D1ComputeMaximumScaleFactor in d2d1.dll is not supported for this application type. SharpDX.Direct2D1.dll calls this API. ◦API CreateDXGIFactory2 in dxgi.dll is not supported for this application type. SharpDX.DXGI.dll calls this API.

Nothing else appears in the audio sample cert report!

Windows Phone on the other hand, did not go so smooth. I tested on a Lumia 920 device, and only managed to get it to compile at launch. Nothing renders (just a black screen), let alone play audio. But that is likely due to rudimentary code modifications I made to get it to run. Specifically, https://github.com/dfkeenan/SharpDX/blob/Toolkit_Audio/Source/Toolkit/SharpDX.Toolkit.Audio/AudioManager.cs#L89 and https://github.com/dfkeenan/SharpDX/blob/Toolkit_Audio/Source/Toolkit/SharpDX.Toolkit.Audio/AudioManager.cs#L97, don't compile with the WP8 project.

[dfkeenan]

@RecursiveCall I have made changes so it compiles under WP8, basically needed to change #if defs to be the same for WP8 as WIN8METRO (, and add the project to the SharpDXWP8 solution). I have pushed these changes up. I have not tested this yet either. I am more focused on getting it API complete/documented than supporting platforms I have no interest in....sorry. Not sure when I will get around to completing this. Though getting it work on WP8 probably a lot more interesting the doing comments lol.

[RecursiveCall]

Adding #if defs for WINDOWS_PHONE were basically my temp changes to get it to build. Those changes don't run on the actual device. I'll pull your latest stuff this week and continue testing with that.

[xoofx]

@RecursiveCall Thanks for the report of certification errors. Will try to fix them this week.

[RecursiveCall]

@dfkeenan let me know when you feel your fork is ready for another run of tests or if there is anything else I can help with.

[dfkeenan]

@RecursiveCall will do. I have re-done pooling, it now tries to pool voices per wave format and SoundEffects now share the 1 pool of SoundEffectInstances. I have added a master volume limiter and started adding reverb effects. Reverb is about 95% there but I have done something wrong with the submix voice. It now crashes if I turn the Reverb option on (and call Apply3D). I was going to have a look today seeing how it's a public holiday but the latest Star Bound patch beckons.....

[ArtiomCiumac]

@dfkeenan, can you, please, create a pull request from your branch? It will be much easier to track the progress this way.

The code looks good - you did a really big job, but it would be nice to add more documentation comments, also it may need some minor code styling improvements.

[dfkeenan]

@ArtiomCiumac , I will create a pull request after I fix the bug with reverb that causes it to crash. I will hopefully be able to do that in the next few days. I worked out what was missing I just have to implement it.

I know the code is a bit scruffy in places and seriously lacking documentation comments. I probably won't get around to that this week.

[ArtiomCiumac]

A pull request doesn't mean an immediate merge. You can create it then add additional commits as needed. The main idea is that it is easier to track the changes and we can discuss the code directly - as in the current issue it is not so handy.

[dfkeenan]

@ArtiomCiumac, aaaaahhh yeah that makes sense, will do.

[dfkeenan]

@ArtiomCiumac I have created a pull request, hope I did it correctly. Let me know if I messed it up. If we are tracking changes in there you can probably close this issue.

[ArtiomCiumac]

Great, let's continue any discussions there. Just for future reference - pull requests better to open against the master_integration branch, and not for master. I should have mentioned this earlier - it's my fault.