cf-gitbot
commented
8 years ago We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/114626393.
Closed youngm closed 8 years ago
cf-gitbot
commented
8 years ago We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/114626393.
emalm
commented
8 years ago Hi, @youngm,
Thanks for the report with all the details! Do you know if this issue started happening with Diego 0.1454.0, or if you simply happened to notice it on that version? Also, what's the corresponding version of CF? 231?
Anyway, the stager component is pretty much only a passthrough for staging requests and responses, and here's the actual code that's writing to staging_info.yml in the staging task container: https://github.com/cloudfoundry-incubator/buildpack_app_lifecycle/blob/master/buildpackrunner/runner.go#L301-L313. Looks to me like it's simply deserializing the bin/release output, picking out values, and re-serializing them, so it could be an issue with the candiedyaml library itself when handling this admittedly complicated scalar value.
Weirdly, when playing around with these files in irb, the ruby 2.1.7 YAML library chokes on both of them: with the Diego version, it reports the same "mapping values" error that you saw, and pinpoints it to the : terminating line 8. With the DEA version, it reports a less specific error, but I can get it to load cleanly by changing each -Dappdynamics.agent.accountAccessKey=''key' to -Dappdynamics.agent.accountAccessKey=''key'' (appending a single quote).
In any case, thanks again, and the Diego team will prioritize it for investigation this coming week.
Best, Eric
youngm
commented
8 years ago @ematpl Here is some more data:
As for the examples I pasted I must have messed the DEA one up while doing some light sanitizing. I've included versions more carefully doctored below.
I'll try to create a test case for you Monday.
DEA
---
buildpack_path: !ruby/object:Pathname
path: /tmp/buildpacks/java-buildpack
detected_buildpack:
start_command: ! 'CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-2.0.1_RELEASE
-memorySizes=permgen:128m..384m -memoryWeights=heap:75,native:15,permgen:5,stack:5
-memoryInitials=heap:5%,permgen:5% -totMemory=$MEMORY_LIMIT) && JAVA_HOME=$PWD/.java-buildpack/open_jdk_jre
JAVA_OPTS="-Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY -javaagent:$PWD/.java-buildpack/app_dynamics_agent/javaagent.jar
-Dappdynamics.agent.applicationName=''appname'' -Dappdynamics.agent.tierName=''CloudFoundryTestExisting''
-Dappdynamics.agent.nodeName=''$(expr $(echo $VCAP_APPLICATION|sed -e ''s/ //g'')
: ''.*application_name.[: ]*.\([a-z_0-9-]\+\).*'')[$(expr $(echo $VCAP_APPLICATION|sed
-e ''s/ //g'') : ''.*instance_index.[: ]*\([0-9]\+\).*'')]-[cf1-dev]'' -Dappdynamics.agent.accountAccessKey=''key''
-Dappdynamics.agent.accountName=''non-prod'' -Dappdynamics.controller.hostName=appd-non-prod1
-Dappdynamics.controller.port=80 $(eval ''if [ -n "$VCAP_DEBUG_MODE" ]; then if
[ "$VCAP_DEBUG_MODE" = "run" ]; then echo "-Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=n";
elif [ "$VCAP_DEBUG_MODE" = "suspend" ]; then echo "-Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=y";
fi fi'') $(eval ''if [ -n "$VCAP_CONSOLE_PORT" ]; then echo "-javaagent:$PWD/.java-buildpack/jmxmp_agent/jmxmp-agent.jar
-Dorg.lds.cloudfoundry.jmxmp.host=$VCAP_CONSOLE_IP -Dorg.lds.cloudfoundry.jmxmp.port=$VCAP_CONSOLE_PORT";
fi'') -server -XX:+UseG1GC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=25 -Djava.awt.headless=true
-Djava.net.preferIPv4Stack=true -Daccess.logging.enabled=false -Dhttp.port=$PORT"
exec $PWD/.java-buildpack/tomcat/bin/catalina.sh run'
effective_procfile:
web: ! 'CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-2.0.1_RELEASE
-memorySizes=permgen:128m..384m -memoryWeights=heap:75,native:15,permgen:5,stack:5
-memoryInitials=heap:5%,permgen:5% -totMemory=$MEMORY_LIMIT) && JAVA_HOME=$PWD/.java-buildpack/open_jdk_jre
JAVA_OPTS="-Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY -javaagent:$PWD/.java-buildpack/app_dynamics_agent/javaagent.jar
-Dappdynamics.agent.applicationName=''appname'' -Dappdynamics.agent.tierName=''CloudFoundryTestExisting''
-Dappdynamics.agent.nodeName=''$(expr $(echo $VCAP_APPLICATION|sed -e ''s/ //g'')
: ''.*application_name.[: ]*.\([a-z_0-9-]\+\).*'')[$(expr $(echo $VCAP_APPLICATION|sed
-e ''s/ //g'') : ''.*instance_index.[: ]*\([0-9]\+\).*'')]-[cf1-dev]'' -Dappdynamics.agent.accountAccessKey=''key''
-Dappdynamics.agent.accountName=''non-prod'' -Dappdynamics.controller.hostName=appd-non-prod1
-Dappdynamics.controller.port=80 $(eval ''if [ -n "$VCAP_DEBUG_MODE" ]; then if
[ "$VCAP_DEBUG_MODE" = "run" ]; then echo "-Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=n";
elif [ "$VCAP_DEBUG_MODE" = "suspend" ]; then echo "-Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=y";
fi fi'') $(eval ''if [ -n "$VCAP_CONSOLE_PORT" ]; then echo "-javaagent:$PWD/.java-buildpack/jmxmp_agent/jmxmp-agent.jar
-Dorg.lds.cloudfoundry.jmxmp.host=$VCAP_CONSOLE_IP -Dorg.lds.cloudfoundry.jmxmp.port=$VCAP_CONSOLE_PORT";
fi'') -server -XX:+UseG1GC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=25 -Djava.awt.headless=true
-Djava.net.preferIPv4Stack=true -Daccess.logging.enabled=false -Dhttp.port=$PORT"
exec $PWD/.java-buildpack/tomcat/bin/catalina.sh run'Diego
detected_buildpack: ""
start_command: CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-2.0.1_RELEASE
-memorySizes=permgen:128m..384m -memoryWeights=heap:75,native:15,permgen:5,stack:5
-memoryInitials=heap:5%,permgen:5% -totMemory=$MEMORY_LIMIT) && JAVA_HOME=$PWD/.java-buildpack/open_jdk_jre
JAVA_OPTS="-Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY -javaagent:$PWD/.java-buildpack/app_dynamics_agent/javaagent.jar
-Dappdynamics.agent.applicationName='appname' -Dappdynamics.agent.tierName='CloudFoundryTestExisting'
-Dappdynamics.agent.nodeName='$(expr $(echo $VCAP_APPLICATION|sed -e 's/ //g') :
'.*application_name.[: ]*.\([a-z_0-9-]\+\).*')[$(expr $(echo $VCAP_APPLICATION|sed
-e 's/ //g') : '.*instance_index.[: ]*\([0-9]\+\).*')]-[cf1-dev]' -Dappdynamics.agent.accountAccessKey='key'
-Dappdynamics.agent.accountName='non-prod' -Dappdynamics.controller.hostName=appd-non-prod1
-Dappdynamics.controller.port=80 $(eval 'if [ -n "$VCAP_DEBUG_MODE" ]; then if [
"$VCAP_DEBUG_MODE" = "run" ]; then echo "-Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=n";
elif [ "$VCAP_DEBUG_MODE" = "suspend" ]; then echo "-Xdebug -Xrunjdwp:transport=dt_socket,address=$VCAP_DEBUG_PORT,server=y,suspend=y";
fi fi') $(eval 'if [ -n "$VCAP_CONSOLE_PORT" ]; then echo "-javaagent:$PWD/.java-buildpack/jmxmp_agent/jmxmp-agent.jar
-Dorg.lds.cloudfoundry.jmxmp.host=$VCAP_CONSOLE_IP -Dorg.lds.cloudfoundry.jmxmp.port=$VCAP_CONSOLE_PORT";
fi') -server -XX:+UseG1GC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=25 -Djava.awt.headless=true
-Djava.net.preferIPv4Stack=true -Daccess.logging.enabled=false -Dhttp.port=$PORT"
exec $PWD/.java-buildpack/tomcat/bin/catalina.sh runFantastic, thanks again, @youngm. A reproduction test case would be really helpful. Also, what's the version of the Java buildpack used to stage the droplet?
Best, Eric
youngm
commented
8 years ago @ematpl It is a heavily customized version of 3.3.1 (hence the complex VCAP_DEBUG_PORT and VCAP_CONSOLE_PORT stuff) with some funky custom app dynamics scripts and such enabled. Though it does work I'm going to try and target something simpler like the binary buildpack for a test case.
youngm
commented
8 years ago Here is a test case that duplicates the problem in my environment: teststaging.tar.gz
To reproduce:
cf push teststaging -b binary_buildpack --no-route -u nonecf enable-diego teststaging & cf restage teststagingcf disable-diego teststagingIf staged on DEA note that it works in both Diego and DEA. Hope it duplicated in your ENV.
emalm
commented
8 years ago Thanks, @youngm, that example looks great. We'll try that test asset out first when the team gets to work on this (almost certainly some time this week).
Best, Eric
emalm
commented
8 years ago Hi, @youngm,
We've fixed this issue in Diego 0.1458.0. Thanks again for the detailed report!
Best, Eric
I haven't yet done enough digging to find the exact component but I thought this would be a good place to start.
I have some buildpacks that produce a
start_commandthat includes a:. The droplet works great on Diego, however, when I attempt to restart to DEA I get the error:mapping values are not allowed in this contextwhen the dea attempts to parse staging_info.yml. I take the same application and stage it using and it works great in both Diego and DEA.I downloaded each of the droplets and the you'll notice that the Diego staged staging_info.yml is invalid. (Warning these staging_info.ymls are pretty crazy but I thought I'd provide the exact test results instead of trimmed down fake results.
DEA staging_info.yml
Diego staging_info.yml