microsoft / pxt-microbit

A Blocks / JavaScript code editor for the micro:bit built on Microsoft MakeCode
https://makecode.microbit.org
Other
721 stars 592 forks source link

"speech" module isn't accessible, I think it should. #4915

Open JayTheAxolotl opened 1 year ago

JayTheAxolotl commented 1 year ago

I'm have Been trying to use the speech module for the class I'm in, however Mu is not available. So we have to use make code which means no speech module. This is really frustrating due to the file for the module no even being downloadable by themselves.

I think someone should add the module under the "advanced" tab, so people that don't have access to a offline IDE can still use the module.

Another two thing that would work would be 1. adding the module as an extension, or 2. make the source code for it downloadable(I get if if that one's not possible)

currently this happens if you try to import the module:

Screen Shot 2023-01-25 at 12 02 36 PM
jaustin commented 1 year ago

@JayTheAxolotl the speech module you're referring to is not available in MakeCode, unfortunately.

You can use the speech module in the online Python editor at https://python.microbit.org/, however this is quite different to MakeCode and more like Mu. For example, there are not blocks and so you can't switch between visual and text-based coding.

Under the hood, MakeCode uses a different technology for running Python code on the micro:bit to Mu, which uses MicroPython. MakeCode Python (which you're using in the screenshot above) is documented here

The MicroPython system is also not event-based in the way MakeCode is, so MakeCode Python code and MicroPython code aren't necessarily interchangeable. For example the way you respond to button presses can be different, and MicroPython has a REPL.

@finneyj and @sehodges I think there was a project in the past to bring speech to MakeCode but I'm not sure about the status of it.

dsssssssss9 commented 1 year ago

@JayTheAxolotl As a workaround I use the Dfplayer mini by DfRobot -https://wiki.dfrobot.com/DFPlayer_Mini_SKU_DFR0299

It is also available from many online suppliers

Although this only plays back prerecorded MP3 files it does have a makecode extension .... https://makecode.microbit.org/pkg/51bit/dfplayermini

Perhaps this may be useful in your projects?

jaustin commented 1 year ago

@jwunderl perhaps something here for factoring into the audio record/playback stuff longer term - I think it should be quite feasible to have extensions with samples. If we can have voice recording on the sim for playback, could we record voice samples that end up as assets in the file?

Something like this that only shows up if you add an audio record/playback extension and that lets you add short PCM clips or something, or record voice.

image

@JohnVidler what was the approach in the sample blocks that you had for sample playback?

adamish commented 1 year ago

I've just finishing making an extension using the SAM C++ code. I'll try and get this submitted soon.

https://github.com/adamish/pxt-billy

image

JayTheAxolotl commented 1 year ago

Wow, after the reply by jaustin I was not expecting this to happen, great job man, really!

finneyj commented 1 year ago

Thanks all. I seem to recall that the major blocker here was actually related to the rather questionable nature of the license associated with the SAM synthesizer?

adamish commented 1 year ago

@finneyj Agree, license situation is dubious. I'm not sure how the BBC microbit micropython project 'get away' with including it?(https://github.com/bbcmicrobit/micropython/tree/master/source/lib/sam) . That project is under MIT.

finneyj commented 1 year ago

Indeed... I wasn't comfortable putting this into the CODAL layer for that reason. Would break my ethical guidelines to put a Lancaster University MIT license on it... But now I guess I don't have to. :)

p.s. Nice job on the technical work that's packaged this into a MakeCode extension @adamish! Integration into the CODAL audio pipeline/mixer looks lovely and clean. :)

adamish commented 1 year ago

I followed the process to submit my extension for this but my ticket 60588 is still being processed.... I asked for update but no response.... misuse of comments I know but anyone here shed any light on this?