protobuf will not compile without thread library

[GoogleCodeExporter]

What steps will reproduce the problem?

1. Compile the library without WIN32 or HAVE_PTHREAD
2. Observe compiler errors

What is the expected output? What do you see instead?

Expect successful compilation, see "No suitable threading library available."

Attached a patch (against 2.3.0) that stubs out the pthread and Mutex stuff if 
you don't configure HAVE_PTHREAD.

Cheers!

Original issue reported on code.google.com by ryan.dra...@gmail.com on 7 Dec 2010 at 5:09

Attachments:

• protobuf_nothreads.patch

[GoogleCodeExporter]

Note, these stubs are enough to compile protobuf-lite. Not sure if more are 
needed for the full library.

Original comment by ryan.dra...@gmail.com on 7 Dec 2010 at 5:11

[GoogleCodeExporter]

Hmm.  I don't think we should automatically fall back to thread-hostile code 
when no threading library is available -- this could cause really hard-to-debug 
problems if it happened by accident.  But we could certainly provide a way for 
the user to explicitly ask for this, e.g. a --without-thread-safety configure 
option.

Original comment by kenton@google.com on 8 Dec 2010 at 3:01

• Changed state: Accepted

[GoogleCodeExporter]

Hmm, what OS/platform are you using? Common platforms should define 
HAVE_PTHREAD.. Maybe it's the acx_pthread.m4 problem..

Original comment by liujisi@google.com on 8 Dec 2010 at 6:27

[GoogleCodeExporter]

Brew mobile platform, with the RVCT4.0 ARM compiler.

I think HAVE_PTHREAD should be around all code blocks that require pthread.h, 
but I agree that you don't want to fall back to unsafe code unless the user 
takes affirmative action during configure.

Perhaps keep the HAVE_PTHREAD define and put it around all the pthread code, 
and then add --without-thread-safety and make it set some kind of 
WITH_STUBBED_THREAD define that would enable the stubs?

Original comment by ryan.dra...@gmail.com on 8 Dec 2010 at 9:08

[GoogleCodeExporter]

For what it's worth, I'm running into this on OS X 10.6.5 with up-to-date 
macports and protobuf svn rev 358.  The configure script does give me a warning:

...
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... -D_THREAD_SAFE
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... no
checking whether -lpthread fixes that... no
checking whether -lc_r fixes that... no
configure: WARNING: Impossible to determine how to use pthreads with shared 
libraries
...

Strange enough, Darwin does come with /usr/lib/libpthread.dylib and 
/usr/include/pthread.h. I'll keep digging, but it's not high on my priority 
list. Maybe just need to tweak the configure.ac a bit?

Original comment by poftware...@gmail.com on 9 Dec 2010 at 4:05

[GoogleCodeExporter]

Looks like r353 broke the m4 code that checks for pthread/sharedlib coexistance 
on OS X. Somehow m4/acx_pthread.m4 ends up injecting -Wl,-z,foo onto the gcc 
command line, which Apple's ld chokes on (invalid option -z or something like 
that).

Reverting to r352 seems to work around the issue.

Original comment by poftware...@gmail.com on 9 Dec 2010 at 4:43

[GoogleCodeExporter]

Yes, the r353 fix isn't correct. Rolled it back at r360. 

Original comment by liujisi@google.com on 9 Dec 2010 at 9:32

[GoogleCodeExporter]

To be clear, the warning printed by configure (for v2.3.0 and earlier) is 
bogus.  It's a side effect of a deeper bug in the m4 file that we're trying to 
fix.  But basically, you can ignore it.

Original comment by kenton@google.com on 15 Dec 2010 at 3:51

[GoogleCodeExporter]

So now I am not sure whether there is a way to build 2.4.0a without pthread 
dependency. I would like to use protobuf in a single-threaded embedded 
environment, where there is no pthread library. Is this possible? If yes, how 
would I invoke "configure" to do this?

Original comment by jonny.de...@gmail.com on 7 Mar 2011 at 8:23

[GoogleCodeExporter]

You will need to modify the protobuf code slightly, but it should be easy.  
Find the places that use pthread (there aren't many of them) and replace them 
with code appropriate for a single-threaded context.  You can just delete the 
mutex lock/unlock calls.  For pthread_once, you'll need to make the "once" type 
just contain a boolean flag and have the once-init call return if the flag is 
already set, otherwise call the callback and then set the flag.

If someone wants to write a patch implementing a --without-thread-safety 
configure option that does this automatically, go for it...

Original comment by kenton@google.com on 8 Mar 2011 at 2:04

[GoogleCodeExporter]

We do have a draft here.... see http://codereview.appspot.com/3540041/
From the review comments, the blocking issue is google/protobuf/stubs/once.h 
needs to include "config.h", which is not acceptable.

You can actually patch the diff to have a hardcoded no thread-safety protobuf.

Original comment by liujisi@google.com on 24 Mar 2011 at 1:14

[GoogleCodeExporter]

Original comment by kenton@google.com on 17 Apr 2011 at 7:47

[GoogleCodeExporter]

Some platforms simply don't have PTHREADS. Some because they aren't 
multithreaded or have unusual OSes (i.e. microcontrollers with RTOS or 
bare-metal, without OS).

Original comment by ronanpaixao@gmail.com on 19 Nov 2011 at 8:59

[GoogleCodeExporter]

I had a look at the proposed patch, the one that cannot be merged into GPB 
release...
Just a thought: once.h is compiled either as part of GPB itself or as part of 
generated pb.cc file. 
I think the compilation of GPB itself can be handled by adding a configure 
option that  will add a definition of 'PROTOBUF_WITHOUT_THREAD_SAFETY' to the 
CC_FLAGS or config.h.
The second scenario, compilation of a generated pb.cc file can be handled by an 
option to the code generator that will define this symbol in the scope of the 
pb.cc file (only).

I know that this approach makes room for erroneous situations like thread-less 
GPB with a thread-enabled generated classes, but I think there must be a way to 
fix this.

is this approach acceptable in any way?
Eyal.

Original comment by Eyal.Far...@gmail.com on 21 Nov 2011 at 7:59

[GoogleCodeExporter]

Maybe one can define weak functions that are thread-safe and then the user may 
override with functions pertaining to each platform. Or separate those 
functions in a single file to make the transition easier.

Original comment by ronanpaixao@gmail.com on 21 Nov 2011 at 1:48

[GoogleCodeExporter]

Original comment by xiaof...@google.com on 4 Feb 2013 at 1:53

• Added labels: Priority-Low
• Removed labels: Priority-Medium

[GoogleCodeExporter]

Has anybody determined the correct way to build protobuf without depending on a 
threading library?  I want to use protobuf on a ARM Cortex-M4 with an RTOS that 
does not provide pthreads.

Do the changes that the original patch made roughly apply to the latest svn 
checkout?

Original comment by hazelnu...@gmail.com on 19 Apr 2013 at 1:02

[GoogleCodeExporter]

On my ARM Cortex-M3 project I just used protobuf-c and replaced the malloc() 
calls with static objects, since only one thread used the objects synchronously.

Original comment by ronanpaixao@gmail.com on 19 Apr 2013 at 2:42

[GoogleCodeExporter]

Is that code up anywhere so I could check it out?  By protobuf-c, you mean this 
one right:
https://code.google.com/p/protobuf-c/

It seems like it hasn't been touched in > 2 years. 

Original comment by hazelnu...@gmail.com on 19 Apr 2013 at 3:31

[GoogleCodeExporter]

Another solution is to use nanopb: http://koti.kapsi.fi/jpa/nanopb/

This worked well for a recent embedded systems project which used PBs.

Original comment by bschlin...@inbound5.com on 23 Apr 2013 at 6:37

[GoogleCodeExporter]

Another solution is to use nanopb: http://koti.kapsi.fi/jpa/nanopb/

This worked well for a recent embedded systems project which used PBs.

Original comment by bschlin...@inbound5.com on 23 Apr 2013 at 6:37