Closed lordadamson closed 3 years ago
If I do a make clean like this:
make -f GNUmakefile-cross clean make clean
it won't delete neon_simd.o
That's weird. What are the permissions on neon_simd.o? ls -Al neon_simd.*
should show them.
This is where I discovered the issue because neon_simd.o was an old version from a previous build
So to reproduce, build iPhone/armv7, then make distclean, and then build iPhone/armv8?
This is a related check-in for GNUmakefile-cross
: commit a783d2bf0742.
It fixes a missing CPPFLAGS
that some folks may use. It does not fix your issue, however.
➜ cryptopp git:(master) ✗ ls -Al neon_simd.*
-rw-r--r-- 1 adam staff 5521 Dec 31 14:47 neon_simd.cpp
-rw-r--r-- 1 adam staff 54132 Jan 1 17:56 neon_simd.o
But it's not different from any other object file. For example:
➜ cryptopp git:(master) ✗ ls -Al integer.*
-rw-r--r-- 1 adam staff 133752 Dec 31 14:47 integer.cpp
-rw-r--r-- 1 adam staff 38699 Dec 31 14:47 integer.h
-rw-r--r-- 1 adam staff 3762732 Jan 1 17:54 integer.o
Here's the output of the clean commands:
➜ cryptopp git:(master) ✗ make -f GNUmakefile-cross clean
Here's what we found... IS_X86: 0, IS_X64: 0, IS_ARM32: 0, IS_ARMV8: 0
rm -f adhoc.cpp.o adhoc.cpp.proto.o cryptlib.o cpu.o integer.o 3way.o adler32.o algebra.o algparam.o allocate.o arc4.o aria.o aria_simd.o ariatab.o asn.o authenc.o base32.o base64.o basecode.o bfinit.o blake2.o blake2b_simd.o blake2s_simd.o blowfish.o blumshub.o camellia.o cast.o casts.o cbcmac.o ccm.o chacha.o chacha_avx.o chacha_simd.o chachapoly.o cham.o cham_simd.o channels.o cmac.o crc.o crc_simd.o darn.o default.o des.o dessp.o dh.o dh2.o dll.o donna_32.o donna_64.o donna_sse.o dsa.o eax.o ec2n.o eccrypto.o ecp.o elgamal.o emsa2.o eprecomp.o esign.o files.o filters.o fips140.o fipstest.o gcm.o gcm_simd.o gf256.o gf2_32.o gf2n.o gf2n_simd.o gfpcrypt.o gost.o gzip.o hc128.o hc256.o hex.o hight.o hmac.o hrtimer.o ida.o idea.o iterhash.o kalyna.o kalynatab.o keccak.o keccak_core.o keccak_simd.o lea.o lea_simd.o luc.o mars.o marss.o md2.o md4.o md5.o misc.o modes.o mqueue.o mqv.o nbtheory.o oaep.o osrng.o padlkrng.o panama.o pkcspad.o poly1305.o polynomi.o pssr.o pubkey.o queue.o rabbit.o rabin.o randpool.o rc2.o rc5.o rc6.o rdrand.o rdtables.o rijndael.o rijndael_simd.o ripemd.o rng.o rsa.o rw.o safer.o salsa.o scrypt.o seal.o seed.o serpent.o sha.o sha3.o sha_simd.o shacal2.o shacal2_simd.o shake.o shark.o sharkbox.o simeck.o simon.o simon128_simd.o skipjack.o sm3.o sm4.o sm4_simd.o sosemanuk.o speck.o speck128_simd.o square.o squaretb.o strciphr.o tea.o tftables.o threefish.o tiger.o tigertab.o ttmac.o tweetnacl.o twofish.o vmac.o wake.o whrlpool.o xed25519.o xtr.o xtrcrypt.o xts.o zdeflate.o zinflate.o zlib.o rdrand-*.o adhoc.o test.o bench1.o bench2.o bench3.o datatest.o dlltest.o fipsalgt.o validat0.o validat1.o validat2.o validat3.o validat4.o validat5.o validat6.o validat7.o validat8.o validat9.o validat10.o regtest1.o regtest2.o regtest3.o regtest4.o
➜ cryptopp git:(master) ✗ make clean
rm -f adhoc.cpp.o adhoc.cpp.proto.o cryptlib.o cpu.o integer.o 3way.o adler32.o algebra.o algparam.o allocate.o arc4.o aria.o aria_simd.o ariatab.o asn.o authenc.o base32.o base64.o basecode.o bfinit.o blake2.o blake2b_simd.o blake2s_simd.o blowfish.o blumshub.o camellia.o cast.o casts.o cbcmac.o ccm.o chacha.o chacha_avx.o chacha_simd.o chachapoly.o cham.o cham_simd.o channels.o cmac.o crc.o crc_simd.o darn.o default.o des.o dessp.o dh.o dh2.o dll.o donna_32.o donna_64.o donna_sse.o dsa.o eax.o ec2n.o eccrypto.o ecp.o elgamal.o emsa2.o eprecomp.o esign.o files.o filters.o fips140.o fipstest.o gcm.o gcm_simd.o gf256.o gf2_32.o gf2n.o gf2n_simd.o gfpcrypt.o gost.o gzip.o hc128.o hc256.o hex.o hight.o hmac.o hrtimer.o ida.o idea.o iterhash.o kalyna.o kalynatab.o keccak.o keccak_core.o keccak_simd.o lea.o lea_simd.o luc.o mars.o marss.o md2.o md4.o md5.o misc.o modes.o mqueue.o mqv.o nbtheory.o oaep.o osrng.o padlkrng.o panama.o pkcspad.o poly1305.o polynomi.o pssr.o pubkey.o queue.o rabbit.o rabin.o randpool.o rc2.o rc5.o rc6.o rdrand.o rdtables.o rijndael.o rijndael_simd.o ripemd.o rng.o rsa.o rw.o safer.o salsa.o scrypt.o seal.o seed.o serpent.o sha.o sha3.o sha_simd.o shacal2.o shacal2_simd.o shake.o shark.o sharkbox.o simeck.o simon.o simon128_simd.o skipjack.o sm3.o sm4.o sm4_simd.o sosemanuk.o speck.o speck128_simd.o square.o squaretb.o sse_simd.o strciphr.o tea.o tftables.o threefish.o tiger.o tigertab.o ttmac.o tweetnacl.o twofish.o vmac.o wake.o whrlpool.o xed25519.o xtr.o xtrcrypt.o xts.o zdeflate.o zinflate.o zlib.o rdrand-*.o adhoc.o test.o bench1.o bench2.o bench3.o datatest.o dlltest.o fipsalgt.o validat0.o validat1.o validat2.o validat3.o validat4.o validat5.o validat6.o validat7.o validat8.o validat9.o validat10.o regtest1.o regtest2.o regtest3.o regtest4.o cryptlib.export.o cpu.export.o integer.export.o 3way.export.o adhoc.export.o adler32.export.o algebra.export.o algparam.export.o allocate.export.o arc4.export.o aria.export.o aria_simd.export.o ariatab.export.o asn.export.o authenc.export.o base32.export.o base64.export.o basecode.export.o bench1.export.o bench2.export.o bench3.export.o bfinit.export.o blake2.export.o blake2b_simd.export.o blake2s_simd.export.o blowfish.export.o blumshub.export.o camellia.export.o cast.export.o casts.export.o cbcmac.export.o ccm.export.o chacha.export.o chacha_avx.export.o chacha_simd.export.o chachapoly.export.o cham.export.o cham_simd.export.o channels.export.o cmac.export.o crc.export.o crc_simd.export.o darn.export.o datatest.export.o default.export.o des.export.o dessp.export.o dh.export.o dh2.export.o dll.export.o dlltest.export.o donna_32.export.o donna_64.export.o donna_sse.export.o dsa.export.o eax.export.o ec2n.export.o eccrypto.export.o ecp.export.o elgamal.export.o emsa2.export.o eprecomp.export.o esign.export.o files.export.o filters.export.o fips140.export.o fipsalgt.export.o fipstest.export.o gcm.export.o gcm_simd.export.o gf256.export.o gf2_32.export.o gf2n.export.o gf2n_simd.export.o gfpcrypt.export.o gost.export.o gzip.export.o hc128.export.o hc256.export.o hex.export.o hight.export.o hmac.export.o hrtimer.export.o ida.export.o idea.export.o iterhash.export.o kalyna.export.o kalynatab.export.o keccak.export.o keccak_core.export.o keccak_simd.export.o lea.export.o lea_simd.export.o luc.export.o mars.export.o marss.export.o md2.export.o md4.export.o md5.export.o misc.export.o modes.export.o mqueue.export.o mqv.export.o nbtheory.export.o oaep.export.o osrng.export.o padlkrng.export.o panama.export.o pkcspad.export.o poly1305.export.o polynomi.export.o pssr.export.o pubkey.export.o queue.export.o rabbit.export.o rabin.export.o randpool.export.o rc2.export.o rc5.export.o rc6.export.o rdrand.export.o rdtables.export.o regtest1.export.o regtest2.export.o regtest3.export.o regtest4.export.o rijndael.export.o rijndael_simd.export.o ripemd.export.o rng.export.o rsa.export.o rw.export.o safer.export.o salsa.export.o scrypt.export.o seal.export.o seed.export.o serpent.export.o sha.export.o sha3.export.o sha_simd.export.o shacal2.export.o shacal2_simd.export.o shake.export.o shark.export.o sharkbox.export.o simeck.export.o simon.export.o simon128_simd.export.o skipjack.export.o sm3.export.o sm4.export.o sm4_simd.export.o sosemanuk.export.o speck.export.o speck128_simd.export.o square.export.o squaretb.export.o sse_simd.export.o strciphr.export.o tea.export.o test.export.o tftables.export.o threefish.export.o tiger.export.o tigertab.export.o ttmac.export.o tweetnacl.export.o twofish.export.o validat0.export.o validat1.export.o validat10.export.o validat2.export.o validat3.export.o validat4.export.o validat5.export.o validat6.export.o validat7.export.o validat8.export.o validat9.export.o vmac.export.o wake.export.o whrlpool.export.o xed25519.export.o xtr.export.o xtrcrypt.export.o xts.export.o zdeflate.export.o zinflate.export.o zlib.export.o cryptlib.import.o cpu.import.o integer.import.o 3way.import.o adler32.import.o algebra.import.o algparam.import.o allocate.import.o arc4.import.o aria.import.o aria_simd.import.o ariatab.import.o asn.import.o authenc.import.o base32.import.o base64.import.o basecode.import.o bfinit.import.o blake2.import.o blake2b_simd.import.o blake2s_simd.import.o blowfish.import.o blumshub.import.o camellia.import.o cast.import.o casts.import.o cbcmac.import.o ccm.import.o chacha.import.o chacha_avx.import.o chacha_simd.import.o chachapoly.import.o cham.import.o cham_simd.import.o channels.import.o cmac.import.o crc.import.o crc_simd.import.o darn.import.o default.import.o des.import.o dessp.import.o dh.import.o dh2.import.o dll.import.o donna_32.import.o donna_64.import.o donna_sse.import.o dsa.import.o eax.import.o ec2n.import.o eccrypto.import.o ecp.import.o elgamal.import.o emsa2.import.o eprecomp.import.o esign.import.o files.import.o filters.import.o fips140.import.o fipstest.import.o gcm.import.o gcm_simd.import.o gf256.import.o gf2_32.import.o gf2n.import.o gf2n_simd.import.o gfpcrypt.import.o gost.import.o gzip.import.o hc128.import.o hc256.import.o hex.import.o hight.import.o hmac.import.o hrtimer.import.o ida.import.o idea.import.o iterhash.import.o kalyna.import.o kalynatab.import.o keccak.import.o keccak_core.import.o keccak_simd.import.o lea.import.o lea_simd.import.o luc.import.o mars.import.o marss.import.o md2.import.o md4.import.o md5.import.o misc.import.o modes.import.o mqueue.import.o mqv.import.o nbtheory.import.o oaep.import.o osrng.import.o padlkrng.import.o panama.import.o pkcspad.import.o poly1305.import.o polynomi.import.o pssr.import.o pubkey.import.o queue.import.o rabbit.import.o rabin.import.o randpool.import.o rc2.import.o rc5.import.o rc6.import.o rdrand.import.o rdtables.import.o rijndael.import.o rijndael_simd.import.o ripemd.import.o rng.import.o rsa.import.o rw.import.o safer.import.o salsa.import.o scrypt.import.o seal.import.o seed.import.o serpent.import.o sha.import.o sha3.import.o sha_simd.import.o shacal2.import.o shacal2_simd.import.o shake.import.o shark.import.o sharkbox.import.o simeck.import.o simon.import.o simon128_simd.import.o skipjack.import.o sm3.import.o sm4.import.o sm4_simd.import.o sosemanuk.import.o speck.import.o speck128_simd.import.o square.import.o squaretb.import.o sse_simd.import.o strciphr.import.o tea.import.o tftables.import.o threefish.import.o tiger.import.o tigertab.import.o ttmac.import.o tweetnacl.import.o twofish.import.o vmac.import.o wake.import.o whrlpool.import.o xed25519.import.o xtr.import.o xtrcrypt.import.o xts.import.o zdeflate.import.o zinflate.import.o zlib.import.o adhoc.import.o test.import.o bench1.import.o bench2.import.o bench3.import.o datatest.import.o dlltest.import.o fipsalgt.import.o validat0.import.o validat1.import.o validat2.import.o validat3.import.o validat4.import.o validat5.import.o validat6.import.o validat7.import.o validat8.import.o validat9.import.o validat10.import.o regtest1.import.o regtest2.import.o regtest3.import.o regtest4.import.o dlltest.dllonly.o
neon_simd.o is missing in both clean commands
OK, I think I see what is going on.
export IOS_SDK=iPhoneOS
export IOS_CPU=arm64
source TestScripts/setenv-ios.sh
export CXXFLAGS="-DNDEBUG -g2 -O3 -fPIC -pipe -fembed-bitcode"
Now, make distclean
. Notice neon_simd.o
is not included in the file list.
We actually have to make -f GNUmakefile-cross distclean
. Notice neon_simd.o
is included in the file list.
The reason the regular makefile running on x86 does not include neon_simd.o
during clean is this piece of goodness in GNUmakefile : 1135
:
# Remove unneeded arch specific files to speed build time.
ifeq ($(IS_PPC32)$(IS_PPC64),00)
SRCS := $(filter-out ppc_%,$(SRCS))
endif
ifeq ($(IS_ARM32)$(IS_ARMV8),00)
SRCS := $(filter-out arm_%,$(SRCS))
SRCS := $(filter-out neon_%,$(SRCS))
endif
ifeq ($(IS_X86)$(IS_X32)$(IS_X64),000)
SRCS := $(filter-out sse_%,$(SRCS))
endif
The source files are filtered from SRCS
, and then the object files are missing from OBJS
:
OBJS := $(SRCS:.cpp=.o)
OBJS := $(OBJS:.S=.o)
Later, make clean
uses LIBOBJS
, DLLOBJS
, etc to clean the artifacts:
.PHONY: clean
clean:
-$(RM) adhoc.cpp.o adhoc.cpp.proto.o $(LIBOBJS) rdrand-*.o $(TESTOBJS) $(DLLOBJS) $(LIBIMPORTOBJS) $(TESTIMPORTOBJS) $(DLLTESTOBJS)
...
aah good catch.
Feels like the kind of bug that is caused by caching. If we're not saving a significant build time with this trick then I'd recommend we get rid of it completely.
It's always nice to have things as simple and straight forward as possible.
I'm wondering if we should just rm *.o
to sidestep it.
In the past I did not want to do it in case there were non-cryptopp object files there.
Ping @mouse07410. Mouse07410, are we allowed to whack all object files in our directory during a make clean
or make distclean
? Or are we limited to just the object files produced by the library?
I think the controlling document is GNU Coding Standards | Standard Targets. The clean and distclean recipes only authorize deleting object files "... that are created by configuring or building the program...".
Honestly rm *.o
is the kind of low-tech solution I'd go for :D :D
OK, this should handle the problem: commit a184ce1aba1a. The commit builds a list of all potential object files and cleans them. It does not filter-out object files. It does not matter which makefile you use.
@lordadamson,
You'll need this one too. I missed it on the original commit. Commit 7315a6f907a1.
Hold on, that broke things. Reverting that one.
@lordadamson,
The difference between rm *.o
and:
CLEAN_SRCS := $(wildcard *.cpp)
CLEAN_OBJS := $(CLEAN_SRCS:.cpp=.o)
is that *.o
whacks all object files. CLEAN_OBJS
only cleans object files produced by the library.
If you compile your own my_test.cxx
, then its object file will be skipped because it is not included in CLEAN_SRCS
. In fact, I use the *.cxx
trick for test programs. Each *.cxx
can have a main
that would otherwise result in duplicate symbols main
if they were included in CLEAN_SRCS
.
This is the commit that handles Cryptogams object files: Commit ae20711372b1.
My personal take: make clean
is allowed to (and should?) remove all the object files in our directory. make distclean
should also remove all the source files that the build (or configuration) process generated.
Thanks @mouse07410,
If this ever surfaces again we'll just whack all of the object files.
Trying to cross-compile to iOS on a macOS
macOS 11.1 Big Sur
Master
This is where I discovered the issue because neon_simd.o was an old version from a previous build
If I do a make clean like this:
it won't delete neon_simd.o
Which leads to all sorts of confusing errors while linking because neon_simd.o is from a previous build.
I understand basic makefiles but I actually don't know how to fix this one because the make file is too advanced and complicated for me lol
If you direct me to where the issue might be I could open a PR with a fix.
For anyone looking for an immediate fix: just delete neon_simd.o and build again.