Issue with complex awk script

jmsfez commented 3 months ago

Dupe Check

[X] I have searched Warp bugs and there are no duplicates

Describe the bug

I have a complex command with an awk script (given in "to reproduce" section). Warp do not succeed to execute it, saying that there is a syntax error. There is no syntax error; I succeed to run it from iTerm. Since I started to switch to Warp, I'd love to avoid such issue 😉 Hope you can help me to make it works

To reproduce

awk -F ',' ' BEGIN {OFS = FS} NR == 1 { for (i=1; i<=NF; i++) { if ($i == "\"TIMESTAMP_DERIVED\"") timestampDerivedIndex = i else if ($i == "\"API_RESOURCE\"" ) apiResourceIndex = i else if ($i == "\"ENTITY_NAME\"" ) entityNameIndex = i else if ($i == "\"API_FAMILY\"" ) apiFamilyIndex = i else if ($i == "\"HTTP_METHOD\"" ) httpMethodIndex = i } print $0, "\"APIResourcePattern\"", "\"Date\"", "\"Operation\"", "\"Information\"" } NR > 1 { date = gensub(/T.Z/, "", "g", $timestampDerivedIndex) apiResourcePattern = gensub(/\/(v?)[0-9]{2}.0-9/, "/\199.0\2", "g", $apiResourceIndex) apiResourcePattern = gensub(/\/[0-9][^\/](\/|$)/ , "/{ID}\1" , "g", apiResourcePattern) apiResourcePattern = gensub(/\/[a-z][0-9][^\/]{5,}(\/|$)/, "/{ID}\1" , "g", apiResourcePattern) apiResourcePattern = gensub(/\/[a-z][^\/]-[^\/](\/|$)/ , "/{ID}\1" , "g", apiResourcePattern) apiResourcePattern = gensub(/\/[^\/]=$/ , "/{ID}" , "g", apiResourcePattern) entityName = $entityNameIndex apiFamily = $apiFamilyIndex httpMethod = $httpMethodIndex operation = "\"Unknown\"" if (apiFamily == "\"SOAP\"") { if (apiResourcePattern ~ /^"create"$/ ) operation = "\"Create\"" else if (apiResourcePattern ~ /^"(query|getUpdated|getDeleted)"$/) operation = "\"Read\"" else if (apiResourcePattern ~ /^"(update|upsert)"$/ ) operation = "\"Update\"" else if (apiResourcePattern ~ /^"delete"$/ ) operation = "\"Delete\"" else if (apiResourcePattern ~ /^"get."$/ ) operation = "\"Read\"" else if (apiResourcePattern ~ /^"(listMetadata|readMetadata)"$/ ) operation = "\"Read\"" } else if (apiFamily == "\"REST\"") { if (httpMethod == "\"GET\"" ) operation = "\"Read\"" else if (httpMethod == "\"(PATCH|PUT)\"" ) operation = "\"Update\"" else if (httpMethod == "\"DELETE\"" ) operation = "\"Delete\"" else if (httpMethod == "\"POST\"" ) { if (apiResourcePattern ~ /^"\/v99.0\/sobjects."$/) operation = "\"Create\"" else if (apiResourcePattern ~ /^"\/v99.0\/composite"$/ ) operation = "\"Composite\"" } } if (entityName != "\" \"") { information = entityName } else { information = "\"Unknown\"" if (apiFamily == "\"SOAP\"") { if (apiResourcePattern ~ /^"(listMetadata|readMetadata)"$/) { information = "\"Entity Metadata\"" } else if (apiResourcePattern ~ /^"getUserInfo"$/ ) { information = "\"User Details\"" } } else if (apiFamily == "\"REST\"") { if (apiResourcePattern ~ /^"\/v99.0\/analytics\/dashboards\/."$/) { information = "\"Analytics\"" } else if (apiResourcePattern ~ /^"\/v99.0\/analytics\/loglines"$/ ) { information = "\"Analytics\"" } else if (apiResourcePattern ~ /^"\/v99.0\/chatter\/users\/(me|{ID})"$/) { information = "\"User Details\"" } else if (apiResourcePattern ~ /^"\/v99.0\/chatter\/."$/ ) { information = "\"Chatter Conversation\"" } else if (apiResourcePattern ~ /^"\/v99.0\/composite"$/ ) { information = "\"Composite Entities\"" } else if (apiResourcePattern ~ /^"\/v99.0\/(sobjects|tooling\/.)"$/ ) { information = "\"Entity Metadata\"" } } } print $0, apiResourcePattern , date, operation, information } ' EventLogFile-{{domain}}-ApiTotalUsage-filtered.csv > EventLogFile-{{domain}}-ApiTotalUsage-filtered-enriched.csv

Expected behavior

File transformation using a complex awk script

Screenshots

Operating system

MacOS

Operating system and version

macOS Sonoma 14.5

Shell Version

GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin23)

Current Warp version

v0.2024.04.02.08.02.stable_01

Regression

No, this bug or issue has existed throughout my experience using Warp

Recent working Warp date

No response

Additional context

No response

Does this block you from using Warp daily?

Yes, this issue prevents me from using Warp daily.

Is this a Warp specific issue? (i.e. does it happen in Terminal, iTerm, Kitty, etc.)

Yes, this I confirmed this only happens in Warp, not other terminals.

Warp Internal (ignore): linear-label:b9d78064-c89e-4973-b153-5178a31ee54e

None

dannyneira commented 3 months ago

Hey @jmsfez I also got syntax errors in iterm and mac terminal. Try the following version:

awk '
{
    apiResourcePattern = gensub(/\/[0-9][^\/]*(\/|$)/, "/{ID}\\1", "g", apiResourcePattern)
    apiResourcePattern = gensub(/\/[a-z][0-9][^\/]{5,}(\/|$)/, "/{ID}\\1", "g", apiResourcePattern)
    apiResourcePattern = gensub(/\/[a-z][^\/]-[^\/]*(\/|$)/, "/{ID}\\1", "g", apiResourcePattern)
    apiResourcePattern = gensub(/[^\/]=$(\/|$)/, "/{ID}", "g", apiResourcePattern)
    entityName = $entityNameIndex
    apiFamily = $apiFamilyIndex
    httpMethod = $httpMethodIndex
    operation = "Unknown"
    if (apiFamily == "SOAP") {
        if (apiResourcePattern ~ /^create$/ ) operation = "Create"
        else if (apiResourcePattern ~ /^(query|getUpdated|getDeleted)$/ ) operation = "Read"
        else if (apiResourcePattern ~ /^(update|upsert)$/ ) operation = "Update"
        else if (apiResourcePattern ~ /^delete$/ ) operation = "Delete"
        else if (apiResourcePattern ~ /^get.*$/ ) operation = "Read"
        else if (apiResourcePattern ~ /^(listMetadata|readMetadata)$/ ) operation = "Read"
    } else if (apiFamily == "REST") {
        if (httpMethod == "GET" ) operation = "Read"
        else if (httpMethod == "PATCH" || httpMethod == "PUT" ) operation = "Update"
        else if (httpMethod == "DELETE" ) operation = "Delete"
        else if (httpMethod == "POST" ) {
            if (apiResourcePattern ~ /^\/v99.0\/sobjects\/$/ ) operation = "Create"
            else if (apiResourcePattern ~ /^\/v99.0\/composite$/ ) operation = "Composite"
        }
    }
    if (entityName != "") {
        information = entityName
    } else {
        information = "Unknown"
        if (apiFamily == "SOAP") {
            if (apiResourcePattern ~ /^(listMetadata|readMetadata)$/) { information = "Entity Metadata" }
            else if (apiResourcePattern ~ /^getUserInfo$/ ) { information = "User Details" }
        } else if (apiFamily == "REST") {
            if (apiResourcePattern ~ /^\/v99.0\/analytics\/dashboards\//) { information = "Analytics" }
            else if (apiResourcePattern ~ /^\/v99.0\/analytics\/loglines$/ ) { information = "Analytics" }
            else if (apiResourcePattern ~ /^\/v99.0\/chatter\/users\/(me|{ID})$/) { information = "User Details" }
            else if (apiResourcePattern ~ /^\/v99.0\/chatter\/$/ ) { information = "Chatter Conversation" }
            else if (apiResourcePattern ~ /^\/v99.0\/composite$/ ) { information = "Composite Entities" }
            else if (apiResourcePattern ~ /^\/v99.0\/(sobjects|tooling\/)$/ ) { information = "Entity Metadata" }
        }
    }
    print $0, apiResourcePattern, date, operation, information
}
' EventLogFile-{{domain}}-ApiTotalUsage-filtered.csv > EventLogFile-{{domain}}-ApiTotalUsage-filtered-enriched.csv

jmsfez commented 3 months ago

Thank for your answer Danny but I'm sorry, it was working on iTerm.

Let's look at other examples where I give you a screen copy of Warp vs iTerm Warp vs iTerm — cURL command.pdf Warp vs iTerm — xmllint command.pdf

Here is the XML file but… 🚨 renamed as a .txt since I can't upload an XML file Activity.txt

As you can see, the commands are complex, and Warp do not recognize the syntax and also have a kind of «bad display effect» (sorry for my english 🙏🏻). We can have a viso if you want to see my screen when I execute the command.

Here are the command as text :

curl -s -X POST \ ${INSTANCE_URL}/services/Soap/m/${API_VERSION} \ -H "Authorization: Bearer ${ACCESS_TOKEN}" \ -H "SOAPAction: readMetadata" \ -H "Content-Type: text/xml; charset=UTF-8" \ -d "$(cat <<EOF <env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tns="http://soap.sforce.com/2006/04/metadata">

${ACCESS_TOKEN} CustomObject Activity EOF)" > Activity.xml cat Activity.xml | xmllint --format - | xmlstarlet sel \ -N env="http://schemas.xmlsoap.org/soap/envelope/" \ -N tns="http://soap.sforce.com/2006/04/metadata" \ -t \ -m "//env:Envelope/env:Body/tns:readMetadataResponse/tns:result/tns:records" \ -v "concat('OBJECT ', tns:fullName, ' \"', tns:label, '\" FIELD Name \"', tns:nameField/tns:label, '\" : ', tns:nameField/tns:type)" -n \ -m "//env:Envelope/env:Body/tns:readMetadataResponse/tns:result/tns:records/tns:fields" \ -v "concat('OBJECT ', ../tns:fullName, ' \"', ../tns:label, '\" FIELD ', tns:fullName, ' \"', tns:label, '\" : ')" \ --if 'tns:type="MasterDetail"' \ -v "concat(tns:type, '(', tns:referenceTo, ', ', tns:relationshipOrder, ')', ' SymetricRelationship(', tns:relationshipName, ', \"', tns:relationshipLabel,'\")')" -n \ --else \ --if 'tns:type="Lookup"' \ -v "concat(tns:type, '(', tns:referenceTo, ')', ' SymetricRelationship(', tns:relationshipName, ', \"', tns:relationshipLabel,'\")')" -n \ --else \ --if 'tns:formula' \ -v "concat('Formula(', tns:type, ')')" -n \ --else \ -v "tns:type" -n

warpdotdev / Warp