Enabling smartphone / mobile device sensors?

[seteahconnoc]

Hi. Which is the easiest way to use smartphone / mobile device sensors in "Snap!"? A platform independent solution for Android, iOS, ... would be great.

[jguille2]

Hi @seteahconnoc , My way (I don't know if it is the easiest one) to do this is using Snap4Arduino web server. I describe this a bit...

• Scratch 1.4 had a remote sensors protocol that mounts a web server. Then, it was different apps (like Physical Sensors for Scratch, Scratch Sensor...) to connect mobiles with Scratch (1.4) and play in our projects with these devices sensors.
• Snap4Arduino implements a webserver with a similar http protocol. Then you can build an app to send and to request data to our Snap projects.
• The http protocol is the same to keep compatibility with many old projects... but unfortunately those mobile apps are not directly applicable, because they use a sensor feature that is only for a sensor block of Scratch 1.4. So you must build your own app.

Joan

[seteahconnoc]

@jguille2 Thanks, that sounds quite promising, but I guess, I 'll still need a little support.

[seteahconnoc]

I found some help. For now, this issue can be closed. I 'll let you know, when I get a result.

[seteahconnoc]

@jguille2 I just found a solution. Pro: It 's quite easy. Contra: It 's a little bit slow and insecure.

I use an ".html" file with JavaScript to get the sensor data of a mobile device (Android or iOS). This ".html" file advices a ".php" file to store the data in a ".txt" file. "Snap!" 's "url" block enables me (via the ".php" file) to read out and use the data in the ".txt" file. All files are saved online. No further configuration is sufficient.

[jguille2]

Ok @seteahconnoc ,

Which library (or middleware) have you used to connect mobile and PC?

If you are running php in your PC (building a webserver) and it connects to your mobile, you can do many things!

Another question (to use at schools) is if this middleware is easily portable. I like Snap4Arduino option because it builds the webserver itself. I write down into my TODO list to make an apk like Physical Sensors for Scratch with a generic Snap! project that allows to make this connection easily.

[seteahconnoc]

I tried different approaches. For now, I am using online (Apache) PHP webspace from a regular provider. I appreciate, that you are expanding the "Snap4Arduino"'s functionality. And I am looking forward to give it another try.

Some time ago, I wrote a PHP script. This file enabled me to build the three blocks "write ... to file ...", "read from file ..." and "delete file ..." using "Snap!"'s own url block. I had to add this line to get things running: header("Access-Control-Allow-Origin: *");

On the pro side: It 's working...

• ... over the internet and not only in the same WiFi network and
• ... independent of the mobile device's operating system.
• The user has not to install an app.
• Students, who understood my mentioned write/read/delete blocks, find it easy to learn about the new mobile device functionality.

On the con side:

• I guess, it 's neither fast nor secure.

For educational purposes, it 's just a nice prototype. Students are quite curious to learn the functionality - which is not to hard to explain. Plus: It helps to create more awareness on what data is sent (without knowing?) by a mobile device. On my TODO list, there would be a nice HTML file with JavaScript code that gets more sensor data.

---

Please react, if I misunderstood your question.

[jguille2]

Ok @seteahconnoc ,

My question was more about the connection between mobile and PC (to get sensor data). In this sense, I see two direct solutions:

• Make an app that uses Snap4Arduino (Scratch 1.4) webservice protocol. Direct and faster connection.
• Or use an IoT solution (app and web service), that allows us to use its own protocol (API) to get sensor data (with url block by an Internet connection in both devices (mobile and PC)).

[seteahconnoc]

As mentioned above, my solution is just a prototype.

As predicted, some students start to get curious and want to improve things. Their first approaches are quite similar to yours. Escpecially Snap4Arduino's web server seems very convenient. Thanks!

[SoerenBusse]

Hey there,

@seteahconnoc asked me if I had an idea how to let a device communicate with Snap4Arduino.

I realised it using your build in NodeJS webserver. In Javascript I read the gyroscope values of the handheld and sends it using JQuerys $.get(...) method to a forward.php script which simply forwards the values to S4A "vars-update" API.

I had used the forward.php to resolve the CORS issue, but as far as I see you've implemented the Allow Cross Origin HTTP header with a "*", so it would be possible to send this request without using a PHP script. But, if I understand it right, the JQuerys $.get() method has some problems with CORS and it's better to use the plain Javascript XMLHTTPRequest which should work to send the commands to the S4A server without a PHP interpreting server.

I've benchmarked the HTTP server using cURL: I created an "When I receive any" block which increments a variable each time it received it. Then I wrote a bash script sending 10 HTTP request with curl to the APIs "broadcast" method with different sleeps between. The best result was 25ms between each request on localhost without loosing commands.

I think, especially if you plan developing an Android App (I'm an Android Developer), it would be better to implement, for example a websockt server. Especially in a local environment websockt should give a huge performance improvement.

So it would be very cool if S4A supports a websockt server soon :)

[brianharvey]

I'm reopening this. The OP has a solution, but as I understand it, this solution is too messy for schools, and we need a really good solution. This would be SO COOL to demo for kids. Every time I see an app inventor demo I'm envious.

[bromagosa]

The bigger issue at hand here is that we can't even type text in some mobile browsers, and it's not our fault :(

Once that works, there are some mobile browser APIs for which we could easily write libraries.

[ToonTalk]

Many sensors work on laptops as well as mobile phones. Here's a project that can report 9 different sensors: https://snap.berkeley.edu/project?user=toontalk&project=sensor%20demo

x acceleration y acceleration z acceleration x axis rotation y axis rotation z axis rotation (compass direction) latitude longitude altitude

SOME OF THE SENSORS WORK ONLY IN CHROME