chriltola / mail

0 stars 0 forks source link

06/08/2024 #38

Open chriltola opened 1 month ago

chriltola commented 1 month ago

this is the function to get second word in line std::string getSecondWord(const std::string& line) { std::istringstream iss(line); std::string word; // Skip the first word iss >> word; // Read first word iss >> word; // Read second word return word; // Return the second word }

how about get back one line? pls write me this function

chriltola commented 1 month ago

To create a function that gets the word before a specific word in a given line, you can modify the logic to search for the desired word and then track the previous word encountered. Here’s a function that accomplishes this:

#include <sstream>
#include <string>

// Function to get the word before the specified word in the line
std::string getWordBefore(const std::string& line, const std::string& targetWord) {
    std::istringstream iss(line);
    std::string word, previousWord;

    while (iss >> word) {
        if (word == targetWord) {
            return previousWord; // Return the word before the target word
        }
        previousWord = word; // Keep track of the previous word
    }
    return ""; // Return empty string if target word is not found or no word before it
}

// Test the function
int main() {
    std::string testLine = "This is a sample line with the target word.";
    std::string target = "target";
    std::string result = getWordBefore(testLine, target);
    std::cout << "The word before '" << target << "' is: " << result << std::endl;

    return 0;
}

Explanation:

  1. String Stream: The function uses std::istringstream to read the line word by word.
  2. Tracking Words: It keeps track of the current word and the previous word.
  3. Check Condition: When the current word matches the targetWord, the function returns the previousWord.
  4. Return Value: If the targetWord is not found or if there is no word before it, the function returns an empty string.

You can call this function with the desired line and target word to get the word preceding the target word.

chriltola commented 1 month ago

Got it, you want to retrieve the line before a specific line in a text file. Here's a function to accomplish that:

#include <fstream>
#include <string>
#include <iostream>

// Function to get the line before a specific line containing a target word
std::string getPreviousLine(const std::string& filePath, const std::string& targetWord) {
    std::ifstream inputFile(filePath);
    std::string line, previousLine;

    if (!inputFile.is_open()) {
        std::cerr << "Error opening file: " << filePath << std::endl;
        return "";
    }

    while (std::getline(inputFile, line)) {
        // Check if the current line contains the target word
        if (line.find(targetWord) != std::string::npos) {
            return previousLine; // Return the previous line
        }
        previousLine = line; // Update the previous line
    }

    inputFile.close();
    return ""; // Return empty string if target word is not found
}

// Test the function
int main() {
    const std::string filePath = "data.txt"; // Replace with your file path
    const std::string targetWord = "TARGET_WORD"; // Replace with the target word to search for

    std::string result = getPreviousLine(filePath, targetWord);
    if (!result.empty()) {
        std::cout << "The line before the line containing '" << targetWord << "' is: " << result << std::endl;
    } else {
        std::cout << "The target word '" << targetWord << "' was not found in the file." << std::endl;
    }

    return 0;
}

Explanation:

  1. Reading the File: The function reads the file line by line using std::getline.
  2. Tracking Lines: It keeps track of the previous line in a variable.
  3. Check Condition: When the current line contains the targetWord, it returns the previous line.
  4. Return Value: If the targetWord is not found, it returns an empty string.

This function should help you retrieve the line before the line containing a specific target word in a text file. You can test and adjust the filePath and targetWord accordingly.

chriltola commented 1 month ago

pls follow the function i provide: // Function to extract the second word from a line std::string getSecondWord(const std::string& line) { std::istringstream iss(line); std::string word; // Skip the first word iss >> word; // Read first word iss >> word; // Read second word return word; // Return the second word }

// Function to extract the Forth word from a line std::string getForthWord(const std::string& line) { std::istringstream iss(line); std::string word; // Skip the first word iss >> word; // Read first word iss >> word; // Read second word iss >> word; // Read third word iss >> word; // Read forth word return word; // Return the second word }

// Function to format the date and time from a line std::string getDateTime(const std::string& line) { std::istringstream iss(line); std::string date, time; iss >> date; // Read DATE iss >> date; // Read the date iss >> time; // Read TIME iss >> time; // Read the time return date + "," + time; // Return formatted date and time }

// Function to extract the ATM number from a line std::string getATM(const std::string& line) { std::string atmPrefix = "ATM:"; if (line.find(atmPrefix) != std::string::npos) { return line.substr(line.find(atmPrefix) + atmPrefix.length()); // Return ATM number } return ""; // Return empty if not found }

chriltola commented 1 month ago

Here's a function to get the previous line from a file, following the style of the functions you've provided:

#include <fstream>
#include <string>
#include <sstream>
#include <iostream>

// Function to get the previous line from a file before a line containing a specific target word
std::string getPreviousLine(const std::string& filePath, const std::string& targetWord) {
    std::ifstream inputFile(filePath);
    std::string line, previousLine;

    if (!inputFile.is_open()) {
        std::cerr << "Error opening file: " << filePath << std::endl;
        return "";
    }

    while (std::getline(inputFile, line)) {
        // Check if the current line contains the target word
        if (line.find(targetWord) != std::string::npos) {
            return previousLine; // Return the previous line
        }
        previousLine = line; // Update the previous line
    }

    inputFile.close();
    return ""; // Return empty string if target word is not found
}

int main() {
    const std::string filePath = "data.txt"; // Replace with your file path
    const std::string targetWord = "TARGET_WORD"; // Replace with the target word to search for

    std::string result = getPreviousLine(filePath, targetWord);
    if (!result.empty()) {
        std::cout << "The line before the line containing '" << targetWord << "' is: " << result << std::endl;
    } else {
        std::cout << "The target word '" << targetWord << "' was not found in the file." << std::endl;
    }

    return 0;
}

Explanation:

  1. File Reading: The function opens the file and reads it line by line using std::getline.
  2. Previous Line Tracking: It keeps track of the previous line in a variable called previousLine.
  3. Condition Check: When the current line contains the targetWord, the function returns the previousLine.
  4. Return Value: If the targetWord is not found, it returns an empty string.

This function should allow you to retrieve the previous line from a text file before a line that contains a specified target word. You can integrate this function into your existing code and adjust the filePath and targetWord as needed.

chriltola commented 1 month ago

from this code:

#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include <set>
#include <sstream>

// Function to extract data between specified lines in the file
std::vector<std::vector<std::string>> extractDataBetweenLines(const std::string& filePath) {
    std::ifstream inputFile(filePath);
    std::vector<std::vector<std::string>> allCapturedData; // Vector to store all sections of captured data
    std::vector<std::string> currentData; // Vector to store the current section of captured data
    std::string line;
    bool capture = false;
    bool hasNotesRetracted = false;
    bool hasE200000000 = false;
    bool hasPOWERINTERRUPTION = false;
    bool hasERR_TOTIMEOUT = false;
    bool hasHOSTTXTIMEOUT = false;

    if (!inputFile.is_open()) {
        std::cerr << "Error opening file: " << filePath << std::endl;
        return allCapturedData;
    }

    while (std::getline(inputFile, line)) {
        // Check if we reached the end of a section
        if (line == "     *PRIMARY CARD READER ACTIVATED*") {
            if (capture) {
                capture = false;
                // If any of the conditions are met, save the current section
                if (hasNotesRetracted || hasE200000000 || hasPOWERINTERRUPTION || hasERR_TOTIMEOUT || hasHOSTTXTIMEOUT) {
                    allCapturedData.push_back(currentData);
                }
                // Clear the current section and reset flags
                currentData.clear();
                hasNotesRetracted = false;
                hasE200000000 = false;
                hasPOWERINTERRUPTION = false;
                hasERR_TOTIMEOUT = false;
                hasHOSTTXTIMEOUT = false;
            }
        }
        else if (line == "------------------------") {
            capture = true; // Start capturing data
        }
        else if (capture) {
            currentData.push_back(line); // Add line to the current section
            // Check for specific conditions within the current section
            if (line == "NOTES RETRACTED") {
                hasNotesRetracted = true;
            }
            if (line.find("E*200000000") != std::string::npos) {
                hasE200000000 = true;
            }
            if (line.find("POWER INTERRUPTION") != std::string::npos) {
                hasPOWERINTERRUPTION = true;
            }
            if (line.find("ERR_TOTIMEOUT") != std::string::npos) {
                hasERR_TOTIMEOUT = true;
            }
            if (line.find("HOST TX TIMEOUT") != std::string::npos) {
                hasHOSTTXTIMEOUT = true;
            }
        }
    }

    inputFile.close();
    return allCapturedData; // Return all captured data sections
}

// Function to extract the second word from a line
std::string getSecondWord(const std::string& line) {
    std::istringstream iss(line);
    std::string word;
    // Skip the first word
    iss >> word; // Read first word
    iss >> word; // Read second word
    return word; // Return the second word
}

// Function to format the date and time from a line
std::string getDateTime(const std::string& line) {
    std::istringstream iss(line);
    std::string date, time;
    iss >> date; // Read DATE
    iss >> date; // Read the date
    iss >> time; // Read TIME
    iss >> time; // Read the time
    return date + "," + time; // Return formatted date and time
}

// Function to extract the ATM number from a line
std::string getATM(const std::string& line) {
    std::string atmPrefix = "ATM:";
    if (line.find(atmPrefix) != std::string::npos) {
        return line.substr(atmPrefix.length()); // Return ATM number
    }
    return ""; // Return empty if not found
}

// Main function
int main() {
    const std::string filePath = "data.txt"; // Replace with your file path
    std::vector<std::vector<std::string>> allData = extractDataBetweenLines(filePath);
    std::set<std::string> printedLines; // Set to track printed lines

    for (const auto& section : allData) {
        for (const auto& line : section) {
            // Determine the message to print for specific problem conditions
            std::string outputLine = line;
            if (line.find("E*200000000") != std::string::npos) {
                outputLine = "ERROR DISPENSER";
            }
            else if (line.find("POWER INTERRUPTION") != std::string::npos) {
                outputLine = "POWER INTERRUPTION DURING DISPENSE";
            }
            else if (line.find("ERR_TOTIMEOUT") != std::string::npos) {
                outputLine = "ERR_TOTIMEOUT";
            }
            else if (line.find("HOST TX TIMEOUT") != std::string::npos) {
                outputLine = "HOST TX TIMEOUT";
            }
            else if (line.find("00APPROVED") != std::string::npos) {
                outputLine = /*previous line */;
            }

            // Only print lines containing specific keywords or problem conditions, ensuring no duplicates
            if (line.find("CARD: ") != std::string::npos) {
                std::string cardNumber = getSecondWord(line); // Get the card number
                if (printedLines.find(cardNumber) == printedLines.end()) {
                    std::cout << cardNumber << ","; // Print the card number
                    printedLines.insert(cardNumber); // Add the line to the set of printed lines
                }
            }
            else if (line.find("TIME ") != std::string::npos) {
                std::string dateTime = getDateTime(line); // Get formatted date and time
                if (printedLines.find(dateTime) == printedLines.end()) {
                    std::cout << dateTime << ","; // Print the formatted date and time
                    printedLines.insert(dateTime); // Add to printed lines
                }
            }
            else if (line.find("Acc:") != std::string::npos) {
                std::string accInfo = line.substr(line.find("Acc:") + 4); // Get account info
                if (printedLines.find(accInfo) == printedLines.end()) {
                    std::cout << accInfo << ","; // Print account info
                    printedLines.insert(accInfo); // Add to printed lines
                }
            }
            else if (line.find("ATM:") != std::string::npos) {
                std::string atmInfo = getATM(line); // Get ATM number
                if (!atmInfo.empty() && printedLines.find(atmInfo) == printedLines.end()) {
                    std::cout << atmInfo << ","; // Print ATM number
                    printedLines.insert(atmInfo); // Add to printed lines
                }
            }
            else if (line.find("USD") != std::string::npos) {
                std::string usdInfo = getATM(line); // Get ATM number
                if (!usdInfo.empty() && printedLines.find(usdInfo) == printedLines.end()) {
                    std::cout << usdInfo << ","; // Print ATM number
                    printedLines.insert(usdInfo); // Add to printed lines
                }
            }
            else if (line.find("KHR") != std::string::npos) {
                std::string khrInfo = getATM(line); // Get ATM number
                if (!khrInfo.empty() && printedLines.find(khrInfo) == printedLines.end()) {
                    std::cout << khrInfo << ","; // Print ATM number
                    printedLines.insert(khrInfo); // Add to printed lines
                }
            }
            else if (outputLine == "NOTES RETRACTED" ||
                outputLine == "ERROR DISPENSER" ||
                outputLine == "POWER INTERRUPTION DURING DISPENSE" ||
                outputLine == "ERR_TOTIMEOUT" ||
                outputLine == "HOST TX TIMEOUT") {
                if (printedLines.find(outputLine) == printedLines.end()) {
                    std::cout << outputLine << ","; // Print the line
                    printedLines.insert(outputLine); // Add the line to the set of printed lines
                }
            }
        }
        std::cout << std::endl; // Separator for each section
    }

    return 0;
}

how to get the previous line on else if (line.find("00APPROVED") != std::string::npos) { outputLine = /previous line /; }

chriltola commented 1 month ago

To get the previous line when "00APPROVED" is found, we need to keep track of the previous line during the iteration. Here's how you can modify your code to achieve that:

#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include <set>
#include <sstream>

// Function to extract data between specified lines in the file
std::vector<std::vector<std::string>> extractDataBetweenLines(const std::string& filePath) {
    std::ifstream inputFile(filePath);
    std::vector<std::vector<std::string>> allCapturedData; // Vector to store all sections of captured data
    std::vector<std::string> currentData; // Vector to store the current section of captured data
    std::string line;
    bool capture = false;
    bool hasNotesRetracted = false;
    bool hasE200000000 = false;
    bool hasPOWERINTERRUPTION = false;
    bool hasERR_TOTIMEOUT = false;
    bool hasHOSTTXTIMEOUT = false;

    if (!inputFile.is_open()) {
        std::cerr << "Error opening file: " << filePath << std::endl;
        return allCapturedData;
    }

    while (std::getline(inputFile, line)) {
        // Check if we reached the end of a section
        if (line == "     *PRIMARY CARD READER ACTIVATED*") {
            if (capture) {
                capture = false;
                // If any of the conditions are met, save the current section
                if (hasNotesRetracted || hasE200000000 || hasPOWERINTERRUPTION || hasERR_TOTIMEOUT || hasHOSTTXTIMEOUT) {
                    allCapturedData.push_back(currentData);
                }
                // Clear the current section and reset flags
                currentData.clear();
                hasNotesRetracted = false;
                hasE200000000 = false;
                hasPOWERINTERRUPTION = false;
                hasERR_TOTIMEOUT = false;
                hasHOSTTXTIMEOUT = false;
            }
        }
        else if (line == "------------------------") {
            capture = true; // Start capturing data
        }
        else if (capture) {
            currentData.push_back(line); // Add line to the current section
            // Check for specific conditions within the current section
            if (line == "NOTES RETRACTED") {
                hasNotesRetracted = true;
            }
            if (line.find("E*200000000") != std::string::npos) {
                hasE200000000 = true;
            }
            if (line.find("POWER INTERRUPTION") != std::string::npos) {
                hasPOWERINTERRUPTION = true;
            }
            if (line.find("ERR_TOTIMEOUT") != std::string::npos) {
                hasERR_TOTIMEOUT = true;
            }
            if (line.find("HOST TX TIMEOUT") != std::string::npos) {
                hasHOSTTXTIMEOUT = true;
            }
        }
    }

    inputFile.close();
    return allCapturedData; // Return all captured data sections
}

// Function to extract the second word from a line
std::string getSecondWord(const std::string& line) {
    std::istringstream iss(line);
    std::string word;
    // Skip the first word
    iss >> word; // Read first word
    iss >> word; // Read second word
    return word; // Return the second word
}

// Function to format the date and time from a line
std::string getDateTime(const std::string& line) {
    std::istringstream iss(line);
    std::string date, time;
    iss >> date; // Read DATE
    iss >> date; // Read the date
    iss >> time; // Read TIME
    iss >> time; // Read the time
    return date + "," + time; // Return formatted date and time
}

// Function to extract the ATM number from a line
std::string getATM(const std::string& line) {
    std::string atmPrefix = "ATM:";
    if (line.find(atmPrefix) != std::string::npos) {
        return line.substr(atmPrefix.length()); // Return ATM number
    }
    return ""; // Return empty if not found
}

// Main function
int main() {
    const std::string filePath = "data.txt"; // Replace with your file path
    std::vector<std::vector<std::string>> allData = extractDataBetweenLines(filePath);
    std::set<std::string> printedLines; // Set to track printed lines

    for (const auto& section : allData) {
        std::string previousLine; // Variable to keep track of the previous line
        for (const auto& line : section) {
            // Determine the message to print for specific problem conditions
            std::string outputLine = line;
            if (line.find("E*200000000") != std::string::npos) {
                outputLine = "ERROR DISPENSER";
            }
            else if (line.find("POWER INTERRUPTION") != std::string::npos) {
                outputLine = "POWER INTERRUPTION DURING DISPENSE";
            }
            else if (line.find("ERR_TOTIMEOUT") != std::string::npos) {
                outputLine = "ERR_TOTIMEOUT";
            }
            else if (line.find("HOST TX TIMEOUT") != std::string::npos) {
                outputLine = "HOST TX TIMEOUT";
            }
            else if (line.find("00APPROVED") != std::string::npos) {
                outputLine = previousLine; // Get the previous line
            }

            // Only print lines containing specific keywords or problem conditions, ensuring no duplicates
            if (line.find("CARD: ") != std::string::npos) {
                std::string cardNumber = getSecondWord(line); // Get the card number
                if (printedLines.find(cardNumber) == printedLines.end()) {
                    std::cout << cardNumber << ","; // Print the card number
                    printedLines.insert(cardNumber); // Add the line to the set of printed lines
                }
            }
            else if (line.find("TIME ") != std::string::npos) {
                std::string dateTime = getDateTime(line); // Get formatted date and time
                if (printedLines.find(dateTime) == printedLines.end()) {
                    std::cout << dateTime << ","; // Print the formatted date and time
                    printedLines.insert(dateTime); // Add to printed lines
                }
            }
            else if (line.find("Acc:") != std::string::npos) {
                std::string accInfo = line.substr(line.find("Acc:") + 4); // Get account info
                if (printedLines.find(accInfo) == printedLines.end()) {
                    std::cout << accInfo << ","; // Print account info
                    printedLines.insert(accInfo); // Add to printed lines
                }
            }
            else if (line.find("ATM:") != std::string::npos) {
                std::string atmInfo = getATM(line); // Get ATM number
                if (!atmInfo.empty() && printedLines.find(atmInfo) == printedLines.end()) {
                    std::cout << atmInfo << ","; // Print ATM number
                    printedLines.insert(atmInfo); // Add to printed lines
                }
            }
            else if (line.find("USD") != std::string::npos) {
                std::string usdInfo = getATM(line); // Get ATM number
                if (!usdInfo.empty() && printedLines.find(usdInfo) == printedLines.end()) {
                    std::cout << usdInfo << ","; // Print ATM number
                    printedLines.insert(usdInfo); // Add to printed lines
                }
            }
            else if (line.find("KHR") != std::string::npos) {
                std::string khrInfo = getATM(line); // Get ATM number
                if (!khrInfo.empty() && printedLines.find(khrInfo) == printedLines.end()) {
                    std::cout << khrInfo << ","; // Print ATM number
                    printedLines.insert(khrInfo); // Add to printed lines
                }
            }
            else if (outputLine == "NOTES RETRACTED" ||
                outputLine == "ERROR DISPENSER" ||
                outputLine == "POWER INTERRUPTION DURING DISPENSE" ||
                outputLine == "ERR_TOTIMEOUT" ||
                outputLine == "HOST TX TIMEOUT") {
                if (printedLines.find(outputLine) == printedLines.end()) {
                    std::cout << outputLine << ","; // Print the line
                    printedLines.insert(outputLine); // Add the line to the set of printed lines
                }
            }

            // Update the previous line
            previousLine = line;
        }
        std::cout << std::endl; // Separator for each section
    }

    return 0;
}

Explanation:

  1. Previous Line Tracking: Added a std::string previousLine variable to keep track of the previous line within the section loop.
  2. Check for "00APPROVED": When the line contains "00APPROVED", outputLine is set to previousLine.
  3. Update Previous Line: After processing each line, the previousLine is updated to the current line.

This ensures that when the code encounters a line with "00APPROVED", it correctly retrieves and uses the previous line.

chriltola commented 1 month ago

edit and below: Currently, there are 4 pending tables issued regarding to the Khmer character and the solution provided by PBB ITD is to extraction issued field into Hex value and perform decode on SAS system. However, SAS system currently support only WLATIN1 and require to change to UTF-8 to support Khmer character. We target to fix this issue on 11/8/24.

chriltola commented 1 month ago

Here’s a revised version of your text:

Currently, there are four pending tables related to the Khmer character issue. The solution provided by PBB ITD is to extract the affected field into a Hex value and decode it within the SAS system. However, the SAS system currently supports only WLATIN1 and needs to be changed to UTF-8 to support Khmer characters. We aim to resolve this issue by 11/8/24.

chriltola commented 1 month ago

i want to do decode hex for field CARDFIELD_VALUE of below proc sql:

proc sql; create view work.W1U2AVS7 as select CARDFIELD_CARD_NO, CARDFIELD_ID, CARDFIELD_VALUE, VERSION, (INPUT(MOD_DATE,yymmdd10.)) as MOD_DATE length = 8 format = Date9. informat = Date9., MOD_USER from work.etls_W1U2AVS7 ; quit;

chriltola commented 1 month ago

To decode a hexadecimal value in SAS, you can use the INPUT() function along with the appropriate informat. If the CARDFIELD_VALUE is a hexadecimal string that needs to be converted to a character or numeric value, you should use the appropriate informat for the target type.

Here’s an example of how you might modify your PROC SQL to decode a hex value in the CARDFIELD_VALUE field:

proc sql;
create view work.W1U2AVS7 as
select
    CARDFIELD_CARD_NO,
    CARDFIELD_ID,
    INPUT(CARDFIELD_VALUE, $hex32.) as CARDFIELD_VALUE_DECODED length = 32,
    VERSION,
    (INPUT(MOD_DATE, yymmdd10.)) as MOD_DATE length = 8 format = Date9. informat = Date9.,
    MOD_USER
from
    work.etls_W1U2AVS7
;
quit;

Explanation:

If the CARDFIELD_VALUE is a numeric value stored in hexadecimal format, you would use an appropriate numeric informat instead (e.g., hex8. for an 8-digit hexadecimal number).

Adjust the length in the length = clause and the informat used in INPUT() as per the specific requirements of your data.

chriltola commented 1 month ago

https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/movefile/n06qan4j3ffr6fn11bs4q11r8r56.htm

chriltola commented 1 month ago
i try to decode this value of E19E8FE19EB6E19F86E19E8420E19E98E19F89E19F81E19E84E19E87E19EBC.
it should give result តាំង ម៉េងជូ
but in result i got only តាំង ម 
chriltola commented 1 month ago

The issue seems to be related to how the hexadecimal value is being decoded and interpreted. When decoding hexadecimal data, especially if it's Unicode (UTF-8 or UTF-16 encoded), it's important to handle the entire string correctly and use the appropriate length and encoding methods.

Given your scenario, if the hexadecimal string represents a UTF-16 encoded string, you should use the $hex. informat to decode the entire string properly. Here’s a more refined approach:

Step-by-Step Solution:

  1. Hexadecimal Conversion: Use INPUT() with the $hex. informat to convert the hex string to a raw binary value.

  2. Encoding Interpretation: After converting the hexadecimal string to raw binary, you need to interpret it as Unicode text (assuming UTF-16 encoding).

Here's how you can adjust your SQL:

proc sql;
    create view work.W1U2AVS7 as
    select
        CARDFIELD_CARD_NO,
        CARDFIELD_ID,
        /* Decoding HEX to UTF-16 and then converting to a readable character format */
        INPUT(CARDFIELD_VALUE, $hex40.) as CARDFIELD_VALUE_DECODED length=20 format=$char20.,
        VERSION,
        (INPUT(MOD_DATE, yymmdd10.)) as MOD_DATE length=8 format=Date9. informat=Date9.,
        MOD_USER
    from 
        work.etls_W1U2AVS7;
quit;

Explanation:

If your data is in a different encoding (like UTF-8), you might need to adjust the decoding method accordingly.

Troubleshooting:

If the output still truncates the data, you might want to check:

  1. String Length: Ensure the decoded string fits within the specified length (20 in this case). Increase it if necessary.

  2. Encoding Consistency: Verify that the hex data is consistently encoded (e.g., all UTF-16 or UTF-8).

If these adjustments don’t resolve the issue, consider inspecting the encoding specifics of the source data to ensure the correct interpretation in SAS.

chriltola commented 1 month ago

this time, i got តាំង ម៉�. actually, our environment, we use UTF-8

chriltola commented 1 month ago

Since your environment uses UTF-8 encoding, the hexadecimal string should be decoded accordingly. Here's how you can handle it:

Step-by-Step Solution:

  1. Hexadecimal to Binary Conversion: Convert the hexadecimal string to raw binary using the $hex. informat.

  2. Binary to UTF-8 Conversion: Use the appropriate character format to interpret the binary data as a UTF-8 encoded string.

Here’s the updated SQL:

proc sql;
    create view work.W1U2AVS7 as
    select
        CARDFIELD_CARD_NO,
        CARDFIELD_ID,
        /* Decoding HEX to UTF-8 and converting to readable text */
        INPUTC(INPUT(CARDFIELD_VALUE, $hex64.), $utf8.) as CARDFIELD_VALUE_DECODED length=32,
        VERSION,
        (INPUT(MOD_DATE, yymmdd10.)) as MOD_DATE length=8 format=Date9. informat=Date9.,
        MOD_USER
    from 
        work.etls_W1U2AVS7;
quit;

Explanation:

Troubleshooting:

This approach should correctly decode and display your UTF-8 encoded string.