gagolews / stringi

Fast and portable character string processing in R (with the Unicode ICU)
https://stringi.gagolewski.com/
Other
304 stars 45 forks source link

TOC overflow with stringi installation #381

Closed bergen288 closed 4 years ago

bergen288 commented 4 years ago

I am stuck with stringi installation on my AIX7.2 server with no internet access. At first, the source file is stringi_1.4.6.tar.gz that has ICUDT issue. Then I downloaded the large stringi-1.4.6.tar.gz with ICUDT data. Unfortunately, I am getting TOC overflow error. According to my google search, it sounds like “g++” needs -bbigtoc option to get big TOC size. But I have trouble to pass this option correctly. Below is my R install statement and important message/error in the log. Can you please help me to address this issue? I also tried with --configure-args="CXX=g++ -maix64 -std=gnu++11 -bbigtoc" (no single quotation marks after g++), but it failed immediately. I also tried --configure-args="STRINGI_CXXFLAGS=-bbigtoc" today, but it failed even earlier.

R CMD INSTALL -l /software/R/site-library --configure-args="CXX='g++ -maix64 -std=gnu++11 -bbigtoc'" pkg-source-files/stringi-1.4.6.tar.gz >stringi.log 2>&1 &

*** stringi configure summary: ICU_FOUND=0 ICUDT_DIR=icu61/data ICU_BUNDLE_VERSION=61 ICUDT_ENDIANNESS=big STRINGI_CXXSTD=CXX_STD=CXX11 STRINGI_CFLAGS=
STRINGI_CPPFLAGS=-I. -Iicu61 -Iicu61/unicode -Iicu61/common -Iicu61/i18n -DU_STATIC_IMPLEMENTATION -DU_COMMON_IMPLEMENTATION -DU_I18N_IMPLEMENTATION -DUCONFIG_USE_LOCAL -UDEBUG -DNDEBUG STRINGI_CXXFLAGS=
STRINGI_LDFLAGS=
STRINGI_LIBS=

*** compiler settings used: CC=gcc -maix64 LD=g++ -maix64 -std=gnu++11 CFLAGS=-O2 -I/opt/freeware/include -I/usr/include -D_GETDELIM
CPPFLAGS=-O2 -I/opt/freeware/include -I/usr/include -D_GETDELIM -UDEBUG -DNDEBUG CXX=g++ -maix64 -std=gnu++11 CXXFLAGS=-O2 -I/opt/freeware/include -I/usr/include -D_GETDELIM
LDFLAGS=
LIBS=

g++ -maix64 -std=gnu++11 -shared -Wl,-brtl -Wl,-G -Wl,-bexpall -Wl,-bnoentry -lc -L/opt/freeware/lib64/R/lib -L/opt/freeware/lib64 -L/opt/freeware/lib -L/usr/lib -o stringi.so stri_brkiter.o stri_collator.o stri_common.o stri_compare.o stri_container_base.o stri_container_bytesearch.o stri_container_listint.o stri_container_listraw.o stri_container_listutf8.o stri_container_regex.o stri_container_usearch.o stri_container_utf16.o stri_container_utf8.o stri_container_utf8_indexable.o stri_encoding_conversion.o stri_encoding_detection.o stri_encoding_management.o stri_escape.o stri_exception.o stri_ICU_settings.o stri_join.o stri_length.o stri_pad.o stri_prepare_arg.o stri_random.o stri_reverse.o stri_search_class_count.o stri_search_class_detect.o stri_search_class_extract.o stri_search_class_locate.o stri_search_class_replace.o stri_search_class_split.o stri_search_class_startsendswith.o stri_search_class_subset.o stri_search_class_trim.o stri_search_common.o stri_search_coll_count.o stri_search_coll_detect.o stri_search_coll_extract.o stri_search_coll_locate.o stri_search_coll_replace.o stri_search_coll_split.o stri_search_coll_startsendswith.o stri_search_coll_subset.o stri_search_boundaries_count.o stri_search_boundaries_extract.o stri_search_boundaries_locate.o stri_search_boundaries_split.o stri_search_fixed_count.o stri_search_fixed_detect.o stri_search_fixed_extract.o stri_search_fixed_locate.o stri_search_fixed_replace.o stri_search_fixed_split.o stri_search_fixed_subset.o stri_search_fixed_startsendswith.o stri_search_in.o stri_search_other_split.o stri_search_regex_count.o stri_search_regex_detect.o stri_search_regex_extract.o stri_search_regex_locate.o stri_search_regex_match.o stri_search_regex_replace.o stri_search_regex_split.o stri_search_regex_subset.o stri_sort.o stri_stats.o stri_stringi.o stri_sub.o stri_test.o stri_time_zone.o stri_time_calendar.o stri_time_symbols.o stri_time_format.o stri_trans_casemap.o stri_trans_other.o stri_trans_normalization.o stri_trans_transliterate.o stri_ucnv.o stri_uloc.o stri_utils.o stri_wrap.o icu61/common/appendable.o icu61/common/bmpset.o icu61/common/brkeng.o icu61/common/brkiter.o icu61/common/bytesinkutil.o icu61/common/bytestream.o icu61/common/bytestrie.o icu61/common/bytestriebuilder.o icu61/common/bytestrieiterator.o icu61/common/caniter.o icu61/common/chariter.o icu61/common/charstr.o icu61/common/cmemory.o icu61/common/cstr.o icu61/common/cstring.o icu61/common/cwchar.o icu61/common/dictbe.o icu61/common/dictionarydata.o icu61/common/dtintrv.o icu61/common/edits.o icu61/common/errorcode.o icu61/common/filteredbrk.o icu61/common/filterednormalizer2.o icu61/common/icudataver.o icu61/common/icuplug.o icu61/common/listformatter.o icu61/common/loadednormalizer2impl.o icu61/common/locavailable.o icu61/common/locbased.o icu61/common/locdispnames.o icu61/common/locdspnm.o icu61/common/locid.o icu61/common/loclikely.o icu61/common/locmap.o icu61/common/locresdata.o icu61/common/locutil.o icu61/common/messagepattern.o icu61/common/normalizer2.o icu61/common/normalizer2impl.o icu61/common/normlzr.o icu61/common/parsepos.o icu61/common/patternprops.o icu61/common/pluralmap.o icu61/common/propname.o icu61/common/propsvec.o icu61/common/punycode.o icu61/common/putil.o icu61/common/rbbi_cache.o icu61/common/rbbi.o icu61/common/rbbidata.o icu61/common/rbbinode.o icu61/common/rbbirb.o icu61/common/rbbiscan.o icu61/common/rbbisetb.o icu61/common/rbbistbl.o icu61/common/rbbitblb.o icu61/common/resbund_cnv.o icu61/common/resbund.o icu61/common/resource.o icu61/common/ruleiter.o icu61/common/schriter.o icu61/common/serv.o icu61/common/servlk.o icu61/common/servlkf.o icu61/common/servls.o icu61/common/servnotf.o icu61/common/servrbf.o icu61/common/servslkf.o icu61/common/sharedobject.o icu61/common/simpleformatter.o icu61/common/stringpiece.o icu61/common/stringtriebuilder.o icu61/common/uarrsort.o icu61/common/ubidi_props.o icu61/common/ubidi.o icu61/common/ubidiln.o icu61/common/ubiditransform.o icu61/common/ubidiwrt.o icu61/common/ubrk.o icu61/common/ucase.o icu61/common/ucasemap_titlecase_brkiter.o icu61/common/ucasemap.o icu61/common/ucat.o icu61/common/uchar.o icu61/common/ucharstrie.o icu61/common/ucharstriebuilder.o icu61/common/ucharstrieiterator.o icu61/common/uchriter.o icu61/common/ucln_cmn.o icu61/common/ucmndata.o icu61/common/ucnv_bld.o icu61/common/ucnv_cb.o icu61/common/ucnv_cnv.o icu61/common/ucnv_ct.o icu61/common/ucnv_err.o icu61/common/ucnv_ext.o icu61/common/ucnv_io.o icu61/common/ucnv_lmb.o icu61/common/ucnv_set.o icu61/common/ucnv_u16.o icu61/common/ucnv_u32.o icu61/common/ucnv_u7.o icu61/common/ucnv_u8.o icu61/common/ucnv.o icu61/common/ucnv2022.o icu61/common/ucnvbocu.o icu61/common/ucnvdisp.o icu61/common/ucnvhz.o icu61/common/ucnvisci.o icu61/common/ucnvlat1.o icu61/common/ucnvmbcs.o icu61/common/ucnvscsu.o icu61/common/ucnvsel.o icu61/common/ucol_swp.o icu61/common/ucurr.o icu61/common/udata.o icu61/common/udatamem.o icu61/common/udataswp.o icu61/common/uenum.o icu61/common/uhash_us.o icu61/common/uhash.o icu61/common/uidna.o icu61/common/uinit.o icu61/common/uinvchar.o icu61/common/uiter.o icu61/common/ulist.o icu61/common/ulistformatter.o icu61/common/uloc_keytype.o icu61/common/uloc_tag.o icu61/common/uloc.o icu61/common/umapfile.o icu61/common/umath.o icu61/common/umutex.o icu61/common/unames.o icu61/common/unifiedcache.o icu61/common/unifilt.o icu61/common/unifunct.o icu61/common/uniset_closure.o icu61/common/uniset_props.o icu61/common/uniset.o icu61/common/unisetspan.o icu61/common/unistr_case_locale.o icu61/common/unistr_case.o icu61/common/unistr_cnv.o icu61/common/unistr_props.o icu61/common/unistr_titlecase_brkiter.o icu61/common/unistr.o icu61/common/unorm.o icu61/common/unormcmp.o icu61/common/uobject.o icu61/common/uprops.o icu61/common/ures_cnv.o icu61/common/uresbund.o icu61/common/uresdata.o icu61/common/usc_impl.o icu61/common/uscript_props.o icu61/common/uscript.o icu61/common/uset_props.o icu61/common/uset.o icu61/common/usetiter.o icu61/common/ushape.o icu61/common/usprep.o icu61/common/ustack.o icu61/common/ustr_cnv.o icu61/common/ustr_titlecase_brkiter.o icu61/common/ustr_wcs.o icu61/common/ustrcase_locale.o icu61/common/ustrcase.o icu61/common/ustrenum.o icu61/common/ustrfmt.o icu61/common/ustring.o icu61/common/ustrtrns.o icu61/common/utext.o icu61/common/utf_impl.o icu61/common/util_props.o icu61/common/util.o icu61/common/utrace.o icu61/common/utrie.o icu61/common/utrie2_builder.o icu61/common/utrie2.o icu61/common/uts46.o icu61/common/utypes.o icu61/common/uvector.o icu61/common/uvectr32.o icu61/common/uvectr64.o icu61/common/wintz.o icu61/i18n/affixpatternparser.o icu61/i18n/alphaindex.o icu61/i18n/anytrans.o icu61/i18n/astro.o icu61/i18n/basictz.o icu61/i18n/bocsu.o icu61/i18n/brktrans.o icu61/i18n/buddhcal.o icu61/i18n/calendar.o icu61/i18n/casetrn.o icu61/i18n/cecal.o icu61/i18n/chnsecal.o icu61/i18n/choicfmt.o icu61/i18n/coleitr.o icu61/i18n/coll.o icu61/i18n/collation.o icu61/i18n/collationbuilder.o icu61/i18n/collationcompare.o icu61/i18n/collationdata.o icu61/i18n/collationdatabuilder.o icu61/i18n/collationdatareader.o icu61/i18n/collationdatawriter.o icu61/i18n/collationfastlatin.o icu61/i18n/collationfastlatinbuilder.o icu61/i18n/collationfcd.o icu61/i18n/collationiterator.o icu61/i18n/collationkeys.o icu61/i18n/collationroot.o icu61/i18n/collationrootelements.o icu61/i18n/collationruleparser.o icu61/i18n/collationsets.o icu61/i18n/collationsettings.o icu61/i18n/collationtailoring.o icu61/i18n/collationweights.o icu61/i18n/compactdecimalformat.o icu61/i18n/coptccal.o icu61/i18n/cpdtrans.o icu61/i18n/csdetect.o icu61/i18n/csmatch.o icu61/i18n/csr2022.o icu61/i18n/csrecog.o icu61/i18n/csrmbcs.o icu61/i18n/csrsbcs.o icu61/i18n/csrucode.o icu61/i18n/csrutf8.o icu61/i18n/curramt.o icu61/i18n/currfmt.o icu61/i18n/currpinf.o icu61/i18n/currunit.o icu61/i18n/dangical.o icu61/i18n/datefmt.o icu61/i18n/dayperiodrules.o icu61/i18n/dcfmtsym.o icu61/i18n/decContext.o icu61/i18n/decfmtst.o icu61/i18n/decimalformatpattern.o icu61/i18n/decimfmt.o icu61/i18n/decimfmtimpl.o icu61/i18n/decNumber.o icu61/i18n/digitaffix.o icu61/i18n/digitaffixesandpadding.o icu61/i18n/digitformatter.o icu61/i18n/digitgrouping.o icu61/i18n/digitinterval.o icu61/i18n/digitlst.o icu61/i18n/double-conversion-bignum-dtoa.o icu61/i18n/double-conversion-bignum.o icu61/i18n/double-conversion-cached-powers.o icu61/i18n/double-conversion-diy-fp.o icu61/i18n/double-conversion-fast-dtoa.o icu61/i18n/double-conversion.o icu61/i18n/dtfmtsym.o icu61/i18n/dtitvfmt.o icu61/i18n/dtitvinf.o icu61/i18n/dtptngen.o icu61/i18n/dtrule.o icu61/i18n/esctrn.o icu61/i18n/ethpccal.o icu61/i18n/fmtable_cnv.o icu61/i18n/fmtable.o icu61/i18n/format.o icu61/i18n/fphdlimp.o icu61/i18n/fpositer.o icu61/i18n/funcrepl.o icu61/i18n/gender.o icu61/i18n/gregocal.o icu61/i18n/gregoimp.o icu61/i18n/hebrwcal.o icu61/i18n/indiancal.o icu61/i18n/inputext.o icu61/i18n/islamcal.o icu61/i18n/japancal.o icu61/i18n/measfmt.o icu61/i18n/measunit.o icu61/i18n/measure.o icu61/i18n/msgfmt.o icu61/i18n/name2uni.o icu61/i18n/nfrs.o icu61/i18n/nfrule.o icu61/i18n/nfsubs.o icu61/i18n/nortrans.o icu61/i18n/nounit.o icu61/i18n/nultrans.o icu61/i18n/number_affixutils.o icu61/i18n/number_compact.o icu61/i18n/number_decimalquantity.o icu61/i18n/number_decimfmtprops.o icu61/i18n/number_fluent.o icu61/i18n/number_formatimpl.o icu61/i18n/number_grouping.o icu61/i18n/number_integerwidth.o icu61/i18n/number_longnames.o icu61/i18n/number_modifiers.o icu61/i18n/number_notation.o icu61/i18n/number_padding.o icu61/i18n/number_patternmodifier.o icu61/i18n/number_patternstring.o icu61/i18n/number_rounding.o icu61/i18n/number_scientific.o icu61/i18n/number_stringbuilder.o icu61/i18n/numfmt.o icu61/i18n/numsys.o icu61/i18n/olsontz.o icu61/i18n/persncal.o icu61/i18n/pluralaffix.o icu61/i18n/plurfmt.o icu61/i18n/plurrule.o icu61/i18n/precision.o icu61/i18n/quant.o icu61/i18n/quantityformatter.o icu61/i18n/rbnf.o icu61/i18n/rbt_data.o icu61/i18n/rbt_pars.o icu61/i18n/rbt_rule.o icu61/i18n/rbt_set.o icu61/i18n/rbt.o icu61/i18n/rbtz.o icu61/i18n/regexcmp.o icu61/i18n/regeximp.o icu61/i18n/regexst.o icu61/i18n/regextxt.o icu61/i18n/region.o icu61/i18n/reldatefmt.o icu61/i18n/reldtfmt.o icu61/i18n/rematch.o icu61/i18n/remtrans.o icu61/i18n/repattrn.o icu61/i18n/rulebasedcollator.o icu61/i18n/scientificnumberformatter.o icu61/i18n/scriptset.o icu61/i18n/search.o icu61/i18n/selfmt.o icu61/i18n/sharedbreakiterator.o icu61/i18n/simpletz.o icu61/i18n/smallintformatter.o icu61/i18n/smpdtfmt.o icu61/i18n/smpdtfst.o icu61/i18n/sortkey.o icu61/i18n/standardplural.o icu61/i18n/strmatch.o icu61/i18n/strrepl.o icu61/i18n/stsearch.o icu61/i18n/taiwncal.o icu61/i18n/timezone.o icu61/i18n/titletrn.o icu61/i18n/tmunit.o icu61/i18n/tmutamt.o icu61/i18n/tmutfmt.o icu61/i18n/tolowtrn.o icu61/i18n/toupptrn.o icu61/i18n/translit.o icu61/i18n/transreg.o icu61/i18n/tridpars.o icu61/i18n/tzfmt.o icu61/i18n/tzgnames.o icu61/i18n/tznames_impl.o icu61/i18n/tznames.o icu61/i18n/tzrule.o icu61/i18n/tztrans.o icu61/i18n/ucal.o icu61/i18n/ucln_in.o icu61/i18n/ucol_res.o icu61/i18n/ucol_sit.o icu61/i18n/ucol.o icu61/i18n/ucoleitr.o icu61/i18n/ucsdet.o icu61/i18n/udat.o icu61/i18n/udateintervalformat.o icu61/i18n/udatpg.o icu61/i18n/ufieldpositer.o icu61/i18n/uitercollationiterator.o icu61/i18n/ulocdata.o icu61/i18n/umsg.o icu61/i18n/unesctrn.o icu61/i18n/uni2name.o icu61/i18n/unum.o icu61/i18n/unumsys.o icu61/i18n/upluralrules.o icu61/i18n/uregex.o icu61/i18n/uregexc.o icu61/i18n/uregion.o icu61/i18n/usearch.o icu61/i18n/uspoof_build.o icu61/i18n/uspoof_conf.o icu61/i18n/uspoof_impl.o icu61/i18n/uspoof.o icu61/i18n/utf16collationiterator.o icu61/i18n/utf8collationiterator.o icu61/i18n/utmscale.o icu61/i18n/utrans.o icu61/i18n/valueformatter.o icu61/i18n/visibledigits.o icu61/i18n/vtzone.o icu61/i18n/vzone.o icu61/i18n/windtfmt.o icu61/i18n/winnmfmt.o icu61/i18n/wintzimpl.o icu61/i18n/zonemeta.o icu61/i18n/zrule.o icu61/i18n/ztrans.o icu61/stubdata/stubdata.o -lm -L/opt/freeware/lib64/R/lib -lR /opt/freeware/lib64/libintl.a ld: 0711-781 ERROR: TOC overflow. TOC size: 91112 Maximum size: 65536 collect2: error: ld returned 12 exit status make: 1254-004 The error code from the last command is 1.

bergen288 commented 4 years ago

I also tried to install stringi with uploaded icudt61b.zip file. However, the job below didn’t try to get ICUDT data from the target zip file. Instead, it still tried to download from internet and hence failed. ICUDT_DIR is the output path instead of input path.

R CMD INSTALL -l /software/R/site-library --configure-vars="ICUDT_DIR=/software/R/icudt/icudt61b.zip" pkg-source-files/stringi1.4.6.tar.gz >stringi.log 2>&1 &

checking whether we can fetch icudt... downloading the ICU data library (icudt) output path: /software/R/icudt/icudt61b.zip/icudt61b.zip download error

gagolews commented 4 years ago

Have you tried compiling ICU from sources? http://site.icu-project.org/

Maybe your OS is shipped with some sort of libicu-dev or libicu-devel package?

bergen288 commented 4 years ago

As I mentioned in my 1st post, I have downloaded your stringi package with ICU so ICU isn't an issue. It's the TOC overflow error.

I noticed that the following flags are ignored on purpose in your configure file so I can't use them to pass "-bbigtoc". CC Purposedly ignored. CFLAGS Purposedly ignored. CPP Purposedly ignored. CPPFLAGS Purposedly ignored. CXX Purposedly ignored. CXXFLAGS Purposedly ignored. LDFLAGS Purposedly ignored. LIBS Purposedly ignored.

Below are other flags: How do I pass "-bbigtoc" in order to avoid TOC overflow? STRINGI_CFLAGS Additional C compiler flags, see also --with-extra-cflags. STRINGI_CPPFLAGS Additional C/C++ preprocessor flags, see also --with-extra-cppflags. STRINGI_CXXFLAGS Additional C++ compiler flags, see also --with-extra-cxxflags. STRINGI_LDFLAGS Additional linker flags, see also --with-extra-ldflags. STRINGI_LIBS Additional libraries to link against, see also --with-extra-libs.

gagolews commented 4 years ago

-bbigtoc seems to be a linker flag.

Could you try --configure-vars="STRINGI_LDFLAGS=-bbigtoc" ?

Or --with-extra-ldflags=-bbigtoc" ?

bergen288 commented 4 years ago

Without --configure-vars, I will get "checking whether the C compiler works... yes" and it will continue until TOC overflow. With either --configure-vars, I will get "checking whether the C compiler works... no" and it stops immediately. Is it possible to modify "configure" file to add "-bbigtoc" manually?

gagolews commented 4 years ago

Yes, you can edit the configure.ac file and then use autoconf to generate configure.

Even better IMO is to edit the Makeconf file, see file.path(R.home("etc"), "Makeconf")

bergen288 commented 4 years ago

The TOC overflow error has been resolved after -Wl,-brtl is replaced with -Wl,-bbigtoc,-brtl in 9 FLAG variables in Makeconf file in R/etc/ subdirectory. The stringi installation has been finished successfully. Thanks a lot.