To the limits of Shoes (MEMORY-ERROR)

[IanTrudel]

Displaying a large amount of data causes a memory error. The following may not be a real world example but it always comes down to how Shoes manages the unexpected. It should raise an exception and allow users to handle it.

require("base64")

Shoes.app do
   para "Opening libshoes..."
   libshoes = Dir["#{DIR}/*"].detect { |n| n =~ /libshoes/ }
   para Base64.encode64(File.open(libshoes, "rb").read)
   para "Done"
end

***MEMORY-ERROR***: cshoes.exe[3528]: GSlice: failed to allocate 4088 bytes (alignment: 4096): Not enough space

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

[ccoupe]

More of a C doesn't have exceptions problem. The author in Gslice decided to print an error but he can return nil or what he has - both of which will crash mysteriously somewhere else or you exit. What would you do?

[IanTrudel]

Using export G_SLICE=always-malloc somehow helps but not really either. It might be a clue. See how to do this at C level here: https://lists.gnu.org/archive/html/emacs-devel/2012-11/msg00371.html

[IanTrudel]

Additional reference: https://developer.gnome.org/glib/stable/glib-running.html

[ccoupe]

If you are truly out of memory, you won't have enough memory to put up a dialog box and process it. shoes.exe is often better at getting fatal error messages. - like the name of the dll that can't be found.

[IanTrudel]

If you are truly out of memory, you won't have enough memory to put up a dialog box and process it.

That would be truly a worst case scenario, such as computer is actually out-of-memory. It's hardly out-of-memory when libshoes.dll is only 80 MB.

shoes.exe is often better at getting fatal error messages.

Some of the changes since two years ago have increased the number of such errors. Not sure what happened in the meantime.

[IanTrudel]

More of a C doesn't have exceptions problem.

The answer to all the prayers. http://feepingcreature.github.io/handling.html

Additional references:

• http://stackoverflow.com/questions/3291382/coming-back-to-life-after-segmentation-violation
• http://stackoverflow.com/questions/8401689/best-practices-for-recovering-from-a-segmentation-fault
• 50,000 gtk buttons? https://github.com/JuliaGraphics/Gtk.jl/issues/131

[dredknight]

After reading this thread I am so confused.... is it going to be a tiny shoes lying inches from my grasp or mega shoes the size of a house miles away from me....

[ccoupe]

Funny. - It would be a big house sliding down the slope towards you. The first hack is clever but truly dangerous because shoes doesn't know the state of ruby.

[IanTrudel]

The first hack is clever but truly dangerous because shoes doesn't know the state of ruby.

The same technique is used in Gnome Bug Buddy to catch segfault, save it to a log file and send it to Gnome Bugzilla bug tracker.

[IanTrudel]

NeverSayDie allows to generate a Ruby exception from segfault.

[ccoupe]

Notice that you have to install system level stuff (sudo apt-get, and port osx) and he doesn't mention Windows. Notice that the author knows what evil it is. Note that Gnome Bug Buddy also so makes assumptions about what is installed on a users system (Linux only, gdb installed and odds are very high it creates a new process in order to report it's findings. It's a massive amount of work to unifify all these things and ship Shoes dll, and OSX and Linux .so's (that may not work on a user's system). The testing effort would be huge.

Not enough benefits for the cost.

[IanTrudel]

Good to get your feedback on this. All valid points. What we can do however is gracefully exit Shoes with raising an exception (such as NeverSayDie) using Gnome Bug Buddy segv_handler approach (stripped down, no GUI, etc) instead of using libsigsegv. My understanding based on both NeverSayDie and segv_handler is that it would be roughly 60 lines of C code to implement this idea.

[ccoupe]

My understanding based on both NeverSayDie and segv_handler is that it would be roughly 60 lines of C code to implement this idea.

And then you need to test it. You need 5 build platforms/boxes and 5 test platforms/boxes and even then you don't know what it really works unless you have more test platforms. Newer linux, newer osx, newer Windows. Plus dealing with gdb (or lldb - ick) on all the build platforms. End result - you get a Shoes Dialog box instead of system dialog box (shoes.exe) instead of a backtrace (cshoes.exe). Guess what? The backtrace is better and neither one of them will tell a naive Shoes user what's gone wrong.

Lots of work, no real benefit.

[IanTrudel]

Why do you put a damper on my enthusiasm? 🐤 Bugs have to be fixed sooner or later. Gotta be open minded.

And then you need to test it. You need 5 build platforms/boxes and 5 test platforms/boxes and even then you don't know what it really works unless you have more test platforms. Newer linux, newer osx, newer Windows.

You have been talking about this a lot lately. How many Shoes 3 users there are? Let's put things into perspective: if the only people who use Shoes daily are you and me, then we should cater to our needs first rather than hypothetical needs. Maintaining 5 platforms is a serious workload. It wouldn't hurt to consider an incremental approach to feature propagation on all platforms.

Guess what? The backtrace is better and neither one of them will tell a naive Shoes user what's gone wrong.

Backtrace is always available to us. There is already a rescue in place to handle exception through shoes_start_exception but the function itself only returns its receiver at the moment.

Plus dealing with gdb (or lldb - ick) on all the build platforms.

gdb is optional — Gnome Bug Buddy uses it to get additional stack information. Catching and handling segfault on its own does not require gdb.

Lots of work, no real benefit.

We are just talking for now. We may be able have a compromise with handling exception in shoes_start_exception. This may help a little. Segfault is just rude. Shoes needs a graceful way to deal with bugs, errors and crashes.

[ccoupe]

If there only a couple of us, using the console then backtrace is sufficient. No? Shoes.exe will display a sort of friendly dialog when a segfault occurs. Doesn't help the newbie running from the menus. If your proposed handler could be written as easily as you say, what could it tell the Shoes.exe user that Windows does not already say when libsrvg or got gtk or ruby faults ? At best, a Ruby backtrace instead of Window's dialog? Would that really help the newbie shoes.exe user? Dead is Dead and there is no way Shoes or Ruby can or should continue. The fact that you can catch a segfault does not mean you can anything cross platform about it. Nice idea. Little practical benefit. Lots of work.

[dredknight]

btw, the better shoes gets the more things I will be able to put in my app. Currently it is used by some 800+ people, from which I have active feedback more or less.

Out of those many downloads I have gotten only one broken shoe which was actually some C++ damaged library. Also if there is some aspect of Shoes that you want to get tested just drop me a line with those and I will see if they can be used in some creative way.

I have been a bit busy with some other stuff recently but things like plots and correlation statistics are on the task list for the app.

As always cheers for the hard work!

[IanTrudel]

Thanks for the feedback @dredknight ! This is invaluable information. We currently know very little to none about what people do with Shoes.

[IanTrudel]

This issue does not crash Shoes on FreeBSD. It displays the content of a large file as expected but Shoes is extremely slow at every step (loading, displaying, scrolling, opening console/IRB, etc). libshoes (shoes.lib on FreeBSD) is roughly 1.4MB. Shoes memory usage went from a typical 57-ish to 254-ish MB.

[ccoupe]

I'm exploring using Rust instead of 'C' for Shoes. This would not automatically fix segfaults or make things go faster but it could help. It appears that we can do this incrementally. Pick something discrete like svg or plots or perhaps main.c and convert it. The C macros are the biggest challenge. I did look at using swift (there are compilers for Linux and Windows and bindings to gtk3) but the C interface is confusing which makes incremental-ism more difficult. Swift would be a natural for OSX but Rust has cross compiler support which I consider a big deal.

[IanTrudel]

This is an interesting proposition. What would be the advantages of moving to Rust rather than using C?

The C macros are to be reworking in any case. There is still clean up to happen. Not against the idea because using Rust might give us the opportunity to improve Shoes in ways that couldn't before but let's make sure it is worthy the efforts.

[pjfitzgibbons]

I’m +1 in this idea as I usually cannot mentally parse straight C on any relatively complex app. Rust encourages easier organization and readability (so I’ve heard).

That said, I might be able to make some time to contribute to a rust attempt.

On Tue, Jul 31, 2018 at 10:41 AM Ian Trudel notifications@github.com wrote:

This is an interesting proposition. What would be the advantages of moving to Rust rather than using C?

The C macros are to be reworking in any case. There is still clean up to happen. Not against the idea because using Rust might give us the opportunity to improve Shoes in ways that couldn't before but let's make sure it is worthy the efforts.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/shoes/shoes3/issues/344#issuecomment-409306691, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAsNpYhig_rs6XQR2sZiF4D5VQ1qSMUks5uMJbFgaJpZM4Mi1do .

-- Peter Fitzgibbons (224) 307-9689

[ccoupe]

Well, Rust is the cool kid on the block, so it's got that going for it. In fact I was just building a newer librsvg 2 on the raspberry pi and it wanted a rust compiler for a couple of files. Since I'm OK in C it's not that important to me but other people feel differently. Plus there are fewer and fewer C folks and those willing to learn it.

There is a project call Helix plus youtube videos describing it, that appears to handle the Ruby macros (Data_Get_Struct) and ruby internal api in a rust-y manner. I don't know the language well enough to say one way or the other. Since Shoes uses the ruby internal api for our classes and objects it shouldn't be too hard to do what helix did for the extra classes.

Plot would be a decent test - its a lot of C and Cairo and not much Ruby api and I know there is a leak/segfault in there somewhere, possibly with pango reference counting so Rust won't fix that, but it might find it. Might not. Then someone could rewrite the Shoes api to Plot to be something more acceptable :-)

It's a lot of learning and work to get from Neat Idea to working.

[ccoupe]

Some Thoughts.

[IanTrudel]

I got to say that Rust does seem an interesting programming language and we could definitively use some of its features (traits, polymorphism, etc).

One thing I always advocated is to move some Ruby code written in C to Ruby and use FFI to connect to the C functions that must be written in C. This would only require rewriting the C functions to mirror the proper method calling. Most of the Ruby classes should be written in Ruby as most of the Ruby code on Shoes C side is just transliterated Ruby code anyway. It would make the use of Rust Helix partially (or entirely) useless.

Using Rust will require a different build environment. The only two maintainers (us) are fully competent in C programming. Rust is a novelty for both of us. We cannot bet on potential helpers by moving to Rust and the overall advantages are really unclear. It also seems to me that we would be better off continuing simplifying the C code base. We did make a lot of progress in 3 years or so.

You could open up a new issue if you wish.