timja
commented
15 years ago changed severity
Open timja opened 15 years ago
timja
commented
15 years ago Problem is result of two settings in codebase resulting in mixed output.
Connecting slave compute via jnlp-url SlaveComputer.class definies charset by
calling Charset.defaultCharset().name
As this slave runs on z/OS, it returns "IBM-1047". This information is used when
start and end of build task is logged - but this happens on windows, so
translated output looks garbled like this:
â£�™£…„@‚¨@¤¢…™@�•–•¨”–¤¢
Forcing SlaveComputer here to tell "ISO-8859-1" start of output turns out then:
Started by user h7daeaz
Output stream of remote shell is logged as-is, but now really IBM-1047 charset
is generated, but written untranslated to log.
Fixed temporary by creation of shell-wrapper where STDOUT and STDERR streams
will be autotranslated to ISO-8859-1
So now whole output becomes usable:
Started by user h7daeaz
Baue auf Slave hosttestu
[test] $ rxsh -xe /tmp/hudson25929.sh
+ set
@="/tmp/hudson25929.sh"
ANT_HOME="/u/home/h7daeaz/ant"
BUILD_ID="2009-06-04_16-51-52"
BUILD_NUMBER="33"
BUILD_TAG="hudson-test-33"
CBCMAIN="/usr/lpp/cbclib/xlc"
DSMI_CONFIG="/usr/lpp/Tivoli/tsm/client/ba/bin/dsm.opt"
DSMI_DIR="/usr/lpp/Tivoli/tsm/client/ba/bin"
DSMI_LOG="/var"
DSM_CONFIG="/usr/lpp/Tivoli/tsm/client/ba/bin/dsm.opt"
DSM_DIR="/usr/lpp/Tivoli/tsm/client/ba/bin"
DSM_LOG="/var"
ERRNO="0"
EXECUTOR_NUMBER="0"
HOME="/u/home/h7daeaz"
HUDSON_COOKIE="ee89e97b-0503-4289-80a2-cbcadf412032"
HUDSON_HOME="D:
Dokumente und Einstellungen\\H7DAEAZ
.hudson"
HUDSON_URL="http://ybpb117886.vkbintra.de:8080/"
IBM_JAVA_COMMAND_LINE="java -jar bin/slave.jar -jnlpUrl
http://ybpb117886.vkbintra.de:8080/computer/hosttestu/slave-agent.jnlp"
IFS="
"
IS_USS="YES"
ITDIR="/usr/local/cvsmnt/CVSTest/toolkit"
JOB_NAME="test"
JSVN="/u/home/h7daeaz/svnkit-1.2.3.5521"
JSVN_JARS="/u/home/h7daeaz/svnkit-1.2.3.5521/svnkit-cli.jar:/u/home/h7daeaz/svnkit-1.2.3.5521/svnkit.jar:/u/home/h7daeaz/svnkit-1.2.3.5521/jna.jar"
LANG="C"
LIBPATH="/usr/lpp/java15/J5.0/bin/classic/libjvm.so:/usr/lpp/java15/J5.0/bin/classic:/usr/lpp/java15/J5.0/bin/:/lib:/usr/lib:/usr/lpp/java/lib::/usr/lpp/JGate/bin/mvs:"
LINENO="1"
LLOPT="Y"
LOGNAME="H7DAEAZ"
MAIL="/usr/mail/H7DAEAZ"
MAILCHECK="600"
MANPATH="/usr/man/%L:/usr/local/man"
NLSPATH="/usr/lib/nls/msg/%L/%N"
NODEPEND="Y"
OPTIND="1"
OS="OS390"
PATH="/usr/lpp/java15/J5.0/bin:/u/home/h7daeaz/bin:/usr/local/bin:/bin:/usr/bin:/usr/lpp/java/bin::/usr/lpp/cbclib/xlc/exe::/usr/lpp/cbclib/xlc/bin:.:/usr/local/cvsmnt/cvs:/usr/local/cvsmnt/CVSTest/toolkit/bin"
PPID="3563"
PS1="\$ "
PS2="> "
PS3="#? "
PS4="+ "
PWD="/u/home/h7daeaz/workspace/test"
RANDOM="27941"
SECONDS="0"
SHELL="/bin/sh"
STEPLIB="SYS1.CBC.SCCNCMP"
TERM="xterm"
TZ="MEZ-1MESZ,M3.5.0,M10.5.0"
WORKSPACE="/u/home/h7daeaz//workspace/test"
_="/usr/lpp/java15/J5.0/bin/java"
_BPXK_AUTOCVT="ON"
_BPXK_CCSIDS="(1047,819)"
_C89_ACCEPTABLE_RC="4"
_C89_CCMODE="1"
_C89_CLIB_PREFIX="SYS1.CBC"
_C89_INCDIRS="/usr/include /usr/lpp/ioclib/include"
_C89_LIBDIRS="/lib /usr/lib"
_C89_PLIB_PREFIX="SYS1.CEE"
_C89_SLIB_PREFIX="SYS1"
_C89_WORK_UNIT="SYSDA"
_CC_CLIB_PREFIX="SYS1.CBC"
_CC_INCDIRS="/usr/include /usr/lpp/ioclib/include"
_CC_LIBDIRS="/lib /usr/lib"
_CC_PLIB_PREFIX="SYS1.CEE"
_CC_SLIB_PREFIX="SYS1"
_CC_WORK_UNIT="SYSDA"
_CREATE_LAYOUT="Y"
_CXX_CLIB_PREFIX="SYS1.CBC"
_CXX_INCDIRS="/usr/include /usr/lpp/ioclib/include"
_CXX_LIBDIRS="/lib /usr/lib"
_CXX_PLIB_PREFIX="SYS1.CEE"
_CXX_SLIB_PREFIX="SYS1"
_CXX_WORK_UNIT="SYSDA"
_EDC_PTHREAD_YIELD="-2"
Finished: SUCCESS
Anyway, it would be nice to have these changes permanently in hudson codebase. I
could do test in this environment.
timja
commented
15 years ago Is anything wrong with description ? Is additional information required ?
timja
commented
12 years ago I have an IBM mainframe that I can test with and I wish to explore this issue further.
timja
commented
10 years ago On the Linux box defaulting to 7 bit ASCII I just export LANG=en_US.UTF-8 before the java slave.jar call to get fancy unicode snowmen in the console output. Would something like this be an option here?
timja
commented
10 years ago Assuming unrelated to the two plugin components.
timja
commented
7 years ago We've encountered the same issue when running Jenkins slave on z/OS.
We used the following workaround to resolve the issue:
"
The workaround is based on z/OS UNIX function called Enhanced ASCII support.
First, you need to have the slave successfully connected and online on z/OS UNIX using the above mentioned java options -Dfile.enconding=ISO8859-1 and -text.
Then, I noticed that when you use the "Execute Shell" build step, Jenkins creates a shell script in my /tmp and tries to execute it using /bin/sh -xe... but this shell script is in ASCII so the text is garbled and the script doesn't execute properly.
What I did was to reroute Jenkins' /tmp directory to my own specified location using java option -Djava.io.tmpdir=/var/jenkins/tmp in the slave configuration.
Then I allocated and mounted a new ZFS data set at /var/jenkins/tmp but defined it to automatically tag all files in the file system as ASCII. An example of a mount command with automatic tagging is:
MOUNT FILESYSTEM('JENKINS.TMP.ZFS')
MOUNTPOINT('/var/jenkins/tmp')
TYPE(ZFS) MODE(RDWR) TAG(TEXT,819)
Finally, prior to the "Execute Shell" build step, I use the "Inject environment variables" build step to inject _BPXK_AUTOCVT=ON before the shell script is executed. This environment variable tells z/OS to automatically convert the ASCII-tagged files from /var/jenkins/tmp to EBCDIC.
The shell script then executes successfully and output is displayed in the console log."
Using Hudson build master on Windows XP together with a slave machine (IBM
mainframe) connected by JNLP.
Slave machine shows up perfectly in system information, e.g.
...
file.encoding IBM-1047
file.separator /
ibm.signalhandling.rs false
ibm.signalhandling.sigchain false
ibm.signalhandling.sigint true
ibm.system.encoding IBM-1047
invokedviajava
java.assistive ON
java.awt.fonts
java.awt.graphicsenv sun.awt.X11GraphicsEnvironment
java.awt.printerjob sun.print.PSPrinterJob
java.class.path slave.jar
java.class.version 49.0
java.compiler j9jit23
java.ext.dirs /usr/lpp/java15/J5.0/lib/ext
java.fullversion J2RE 1.5.0 IBM J9 2.3 z/OS s390-31 j9vmmz3123-20080315 (JIT
enabled) J9VM - 20080314_17962_bHdSMr JIT - 20080130_0718ifx2_r8 GC - 200802_08
java.home /usr/lpp/java15/J5.0
...
When running shell command on such a slave, charset (IBM-1047) seems to be
ignored, so output of shell is unreadable in web interface:
â£�™£...„@‚¨@¤¢...™@�•
•¨"¤¢Baue auf Slave hosttestu
[FreeStyle] $ sh -xe /tmp/hudson38047.sh
N@¢...£@ |~a£"—aˆ¤„¢-•óøðô÷K¢ˆ ÂäÉÓÄmÉÄ~òððù`ðõ`ðõmññ`ðô`ñö ÂäÉÓÄmÕäÔÂÅÙ~òö ÂäÉÓÄmãÁÇ~ˆ¤„¢-•`Æ™......⣨"...`òö ÃÂÃÔÁÉÕ~a¤¢™a"——aƒ‚ƒ"‰‚a§"ƒ ÄâÔÉmÃÖÕÆÉÇ~a¤¢™a"——a㉥
"‰a£¢"aƒ"‰...•£a‚�a‚‰•a„¢"K—£ ÄâÔÉmÄÉÙ~a¤¢™a"——a㉥-"‰a£¢"aƒ"‰...•£a‚�a‚‰• ÄâÔÉmÓÖÇ~a¥�™ ÄâÔmÃÖÕÆÉÇ~a¤¢™a"——a㉥"‰a£¢"aƒ"‰...•£a‚�a‚‰•a„¢"K—£ ÄâÔmÄÉÙ~a¤¢™a"——a㉥-"‰a£¢"aƒ"‰...•£a‚�a‚‰• ÄâÔmÓÖÇ~a¥�™ ÅÙÙÕÖ~ð ÅçÅÃäãÖÙmÕäÔÂÅÙ~ð ÈÖÔÅ~a¤aˆ-"...aˆ÷„�...�© ÈäÄâÖÕmÃÖÖÒÉÅ~ùƒù„ð...ò...`ø÷öõ`ôöññ`‚‚ƒø`ðùñ‚ñò÷òõööù ÈäÄâÖÕmÈÖÔÅ~ÄzààÄ-'¤"...•£...@¤•„@ʼn•¢£...""¤•‡...•ààÈ÷ÄÁÅÁéààKˆ¤„¢-• ÈäÄâÖÕmäÙÓ~ˆ££—zaa¨‚£"öðøñôùK¥'‚‰•£™�K„...zøðøða ÉÂÔmÑÁåÁmÃÖÔÔÁÕÄmÓÉÕÅ~'�¥�@`'�™@¢"�¥...K'�™@`'•"—ä™"@ˆ££—zaa¨‚£"öðøñôùK¥'‚‰•£™�K„...zøðøðaƒ-"—¤£...™aˆ-¢££...¢£¤a¢"�¥...`�‡...•£K'•"— ÉÆâ~@ Éâmäââ~èÅâ ÉãÄÉÙ~a¤¢™a"ƒ�"aƒ¥¢"•£aÃåâã...¢£a£-"'‰£ ÑÖÂmÕÁÔÅ~Æ™......⣨"... ÑâåÕ~a¤aˆ-"...aˆ÷„�...�©a¢¥•'‰£`ñKòKóKõõòñ ÑâåÕmÑÁÙâ~a¤aˆ-"...aˆ÷„�...�©a¢¥•'‰£`ñKòKóKõõòña¢¥•'‰£`ƒ"‰K'�™za¤aˆ-"...aˆ÷„�...�©a¢¥•'‰£`ñKòKóKõõòña¢¥•'‰£K'�™za¤aˆ-"...aˆ÷„�...�©a¢¥•'‰£`ñKòKóKõõòña'•�K'�™ ÓÁÕÇ~à ÓÉÂ×ÁãÈ~a¤¢™a"——a'�¥�ñõaÑõKða‚‰•aƒ"�¢¢‰ƒa"‰‚'¥"K¢-za¤¢™a"——a'�¥�ñõaÑõKða‚‰•aƒ"�¢¢‰ƒza¤¢™a"——a'�¥�ñõaÑõKða‚‰•aza"‰‚za¤¢™a"‰‚za¤¢™a"——a'�¥�a"‰‚zza¤¢™a"——aÑÇ�£...a‚‰•a"¥¢z ÓÉÕÅÕÖ~ñ ÓÓÖ×ã~è ÓÖÇÕÁÔÅ~È÷ÄÁÅÁé ÔÁÉÓ~a¤¢™a"�‰"aÈ÷ÄÁÅÁé ÔÁÉÓÃÈÅÃÒ~öðð ÔÁÕ×ÁãÈ~a¤¢™a"�•alÓza¤¢™a"-ƒ�"a"�• ÕÓâ×ÁãÈ~a¤¢™a"‰‚a•"¢a"¢‡alÓalÕ ÕÖÄÅ×ÅÕÄ~è Ö×ãÉÕÄ~ñ Öâ~Öâóùð ×ÁãÈ~a¤¢™a"——a'�¥�ñõaÑõKða‚‰•za¤¢™a"ƒ�"a‚‰•za‚‰•za¤¢™a‚‰•za¤¢™a"——a'�¥�a‚‰•zza¤¢™a"——aƒ‚ƒ"‰‚a§"ƒa...§...zza¤¢™a"——aƒ‚ƒ"‰‚a§"ƒa‚‰•zKza¤¢™a"-ƒ�"aƒ¥¢"•£aƒ¥¢za¤¢™a"-ƒ�"aƒ¥¢"•£aÃåâã...¢£a£-"'‰£a‚‰• ××ÉÄ~øóøøùõó÷ ×âñ~à[@ ×âò~n@ ×âó~{o@ ×âô~N@ ×æÄ~a¤aˆ-"...aˆ÷„�...�©aˆ¤„¢•a¦™'¢—�ƒ...aÆ™......⣨"... ÙÁÕÄÖÔ~òøõðõ âÅÃÖÕÄâ~ð âÈÅÓÓ~a‚‰•a¢ˆ âãÅ×ÓÉÂ~âèâñKÃÂÃKâÃÃÕÃÔ× ãÅÙÔ~§£...™" ãé~ÔÅé`ñÔÅâékÔóKõKðkÔñðKõKð æÖÙÒâ×ÁÃÅ~a¤aˆ-"...aˆ÷„�...�©aˆ¤„¢•a¦™'¢—�ƒ...aÆ™......⣨"... m~a¤¢™a"——a'�¥�ñõaÑõKða‚‰•a'�¥� mÂ×çÒmÁäãÖÃåã~ÖÕ mÃøùmÁÃÃÅ×ãÁÂÓÅmÙÃ~ô mÃøùmÃÃÔÖÄÅ~ñ mÃøùmÃÓÉÂm×ÙÅÆÉç~âèâñKÃÂà mÃøùmÉÕÃÄÉÙâ~a¤¢™a‰•ƒ"¤„...@a¤¢™a"——a‰-ƒ"‰‚a‰•ƒ"¤„... mÃøùmÓÉÂÄÉÙâ~a"‰‚@a¤¢™a"‰‚ mÃøùm×ÓÉÂm×ÙÅÆÉç~âèâñKÃÅÅ mÃøùmâÓÉÂm×ÙÅÆÉç~âèâñ mÃøùmæÖÙÒmäÕÉã~âèâÄÁ mÃÃmÃÓÉÂm×ÙÅÆÉç~âèâñKÃÂà mÃÃmÉÕÃÄÉÙâ~a¤¢™a‰•ƒ"¤„...@a¤¢™a"——a‰-ƒ"‰‚a‰•ƒ"¤„... mÃÃmÓÉÂÄÉÙâ~a"‰‚@a¤¢™a"‰‚ mÃÃm×ÓÉÂm×ÙÅÆÉç~âèâñKÃÅÅ mÃÃmâÓÉÂm×ÙÅÆÉç~âèâñ mÃÃmæÖÙÒmäÕÉã~âèâÄÁ mÃÙÅÁãÅmÓÁèÖäã~è mÃççmÃÓÉÂm×ÙÅÆÉç~âèâñKÃÂà mÃççmÉÕÃÄÉÙâ~a¤¢™a‰•ƒ"¤„...@a¤¢™a"——a‰-ƒ"‰‚a‰•ƒ"¤„... mÃççmÓÉÂÄÉÙâ~a"‰‚@a¤¢™a"‰‚ mÃççm×ÓÉÂm×ÙÅÆÉç~âèâñKÃÅÅ mÃççmâÓÉÂm×ÙÅÆÉç~âèâñ mÃççmæÖÙÒmäÕÉã~âèâÄÁ mÅÄÃm×ãÈÙÅÁÄmèÉÅÓÄ~`ò Ɖ•‰¢ˆ...„z@âäÃÃÅââLast line is output of unix build-in shell command "set", manually converted to
ASCII it looks like this:
+ set
...@="/tmp/hudson38047.sh"...BUILD_ID="2009-05-05_11-04-16"...BUILD_NUMBER="26"...BUILD_TAG="hudson-FreeStyle-26"...CBCMAIN="/usr/lpp/cbclib/xlc"...DSMI_CONFIG="/usr/lpp/Tivoli/tsm/client/b /bin/dsm.opt"...DSMI_DIR="/usr/lpp/Tivoli/tsm/client/b /bin"...DSMI_LOG="/v r"...DSM_CONFIG="/usr/lpp/Tivoli/tsm/client/b /bin/dsm.opt"...DSM_DIR="/usr/lpp/Tivoli/tsm/client/b /bin"...DSM_LOG="/v r"...ERRNO="0"...EXECUTOR_NUMBER="0"...HOME="/u/home/h7d e z"...HUDSON_COOKIE="9c9d0e2e-8765-4611-bbc8-091b12725669"...HUDSON_HOME="D:
Dokumente
und
Einstellungen\\H7DAEAZ
.hudson"...HUDSON_URL="http://ybtm608149.vkbintr .de:8080/"...IBM_JAVA_COMMAND_LINE="j v
-j r sl ve.j r -jnlpUrl
http://ybtm608149.vkbintr .de:8080/computer/hosttestu/sl ve- gent.jnlp"...IFS=" ..."...IS_USS="YES"...ITDIR="/usr/loc l/cvsmnt/CVSTest/toolkit"...JOB_NAME="FreeStyle"...JSVN="/u/home/h7d e z/svnkit-1.2.3.5521"...JSVN_JARS="/u/home/h7d e z/svnkit-1.2.3.5521/svnkit-cli.j r:/u/home/h7d e z/svnkit-1.2.3.5521/svnkit.j r:/u/home/h7d e z/svnkit-1.2.3.5521/jn .j r"...LANG="C"...LIBPATH="/usr/lpp/j v 15/J5.0/bin/cl ssic/libjvm.so:/usr/lpp/j v 15/J5.0/bin/cl ssic:/usr/lpp/j v 15/J5.0/bin/:/lib:/usr/lib:/usr/lpp/j v /lib::/usr/lpp/JG te/bin/mvs:"...LINENO="1"...LLOPT="Y"...LOGNAME="H7DAEAZ"...MAIL="/usr/m il/H7DAEAZ"...MAILCHECK="600"...MANPATH="/usr/m n/%L:/usr/loc l/m n"...NLSPATH="/usr/lib/nls/msg/%L/%N"...NODEPEND="Y"...OPTIND="1"...OS="OS390"...PATH="/usr/lpp/j v 15/J5.0/bin:/usr/loc l/bin:/bin:/usr/bin:/usr/lpp/j v /bin::/usr/lpp/cbclib/xlc/exe::/usr/lpp/cbclib/xlc/bin:.:/usr/loc l/cvsmnt/cvs:/usr/loc l/cvsmnt/CVSTest/toolkit/bin"...PPID="83889537"...PS1="\$
"...PS2="> "...PS3="#? "...PS4="+
"...PWD="/u/home/h7d e z/hudson/worksp ce/FreeStyle"...RANDOM="28505"...SECONDS="0"...SHELL="/bin/sh"...STEPLIB="SYS1.CBC.SCCNCMP"...TERM="xterm"...TZ="MEZ-1MESZ,M3.5.0,M10.5.0"...WORKSPACE="/u/home/h7d e z/hudson/worksp ce/FreeStyle"..._="/usr/lpp/j v 15/J5.0/bin/j v "..._BPXK_AUTOCVT="ON"..._C89_ACCEPTABLE_RC="4"..._C89_CCMODE="1"..._C89_CLIB_PREFIX="SYS1.CBC"..._C89_INCDIRS="/usr/include
/usr/lpp/ioclib/include"..._C89_LIBDIRS="/lib
/usr/lib"..._C89_PLIB_PREFIX="SYS1.CEE"..._C89_SLIB_PREFIX="SYS1"..._C89_WORK_UNIT="SYSDA"..._CC_CLIB_PREFIX="SYS1.CBC"..._CC_INCDIRS="/usr/include
/usr/lpp/ioclib/include"..._CC_LIBDIRS="/lib
/usr/lib"..._CC_PLIB_PREFIX="SYS1.CEE"..._CC_SLIB_PREFIX="SYS1"..._CC_WORK_UNIT="SYSDA"..._CREATE_LAYOUT="Y"..._CXX_CLIB_PREFIX="SYS1.CBC"..._CXX_INCDIRS="/usr/include
/usr/lpp/ioclib/include"..._CXX_LIBDIRS="/lib
/usr/lib"..._CXX_PLIB_PREFIX="SYS1.CEE"..._CXX_SLIB_PREFIX="SYS1"..._CXX_WORK_UNIT="SYSDA"..._EDC_PTHREAD_YIELD="-2"...Finished:
SUCCESS
Since charset information of slave is available, output should be (configuration
option?) converted to e.g. UTF-8 for web browsing.
Originally reported by olaf_vkb, imported from: Charset of slave builder should be considered / changeable on slave definition