ahaydon / Qlik-Cli-Windows

Qlik-Cli for Windows
https://adamhaydon.uk/Qlik-Cli-Windows/
GNU General Public License v2.0
110 stars 51 forks source link

[Help Wanted] Is there a way to remove content from a content library? #169

Closed aburnsy closed 2 years ago

aburnsy commented 2 years ago

Or alternatively a way of syncing a Qlik content library with a local version.

levi-turner commented 2 years ago

A quick and dirty method:

$contentLibraryName = 'foo' # Specify content Library Name
$localPath = 'C:\tmp\contentLibrarySync\' # Specify local sync path
$localFiles = gci -Path $localPath # Get the local files

$null = Connect-Qlik
$contentLibraries = Get-QlikContentLibrary
$contentLibrary = $contentLibraries | Where-Object {$_.Name -eq $contentLibraryName}
$contentLibrary = Get-QlikContentLibrary -id $($contentLibrary.id)
$contentLibraryFiles = $contentLibrary.references

Set-Location $localPath

$contentLibraryFiles | % {
    # Change /content/foo/file.png --> file.png
    $file = $_.externalPath -replace "\/content\/$($contentLibraryName)\/"
    # Check to see if file exists locally
    $fileCheck = Get-ChildItem -Path "$($localPath)$($file)" -ErrorAction SilentlyContinue
    if(!$fileCheck) {
        # File in content library but not on disk; delete these files
        $null = Invoke-QlikDelete -path "/qrs/ContentLibrary/$($contentLibraryName)/deletecontent?externalpath=$($file)"
    } else {
        # File exists on disk and in content library
    }
    # foreach local file, upload and replace (if present)
    $localFiles | % { 
        $null = Import-QlikContent -LibraryName $contentLibraryName -FilePath "$($_.Name)" -Overwrite:$true
    }
}
aburnsy commented 2 years ago

@levi-turner thanks very much! I couldn't figure how the syntax worked for the deletecontent piece.

aburnsy commented 2 years ago

@levi-turner FYI on versions > Sept 2020, this will fail with 429 errors. I'm asking Qlik for details on the DoS configuration that is in place. I found the link below, but even throttling on my side with only 1 request every 10 seconds doesn't work. I don't want to turn off that feature entirely.

https://community.qlik.com/t5/Knowledge/Qlik-Sense-QRS-API-Error-429-in-September-2020-and-later-when/ta-p/1798659

levi-turner commented 2 years ago

I'd encourage you to work through Qlik Support for clarification on the product limitations. The doc you found does not seem quite right on my November 2021 installation. On my end, I was only able to run the 8 count iteration with this test script:

$contentLibraryName = 'foo' # Specify content Library Name
$localPath = 'C:\tmp\sync' # Specify local sync path
$i = 1

1..30 | % {
    Write-Host "Begin $($i) test"
    Get-ChildItem -Path $localPath -Include *.* -File -Recurse | foreach { $_.Delete()}
    1..$i | % {
        Copy-Item "C:\tmp\file.png" -Destination "C:\tmp\sync\file_$($_).png"
    }
    $null = Connect-Qlik
    $contentLibraries = Get-QlikContentLibrary
    $contentLibrary = $contentLibraries | Where-Object {$_.Name -eq $contentLibraryName}
    $contentLibrary = Get-QlikContentLibrary -id $($contentLibrary.id)
    $contentLibraryFiles = $contentLibrary.references
    Set-Location $localPath
    $contentLibraryFiles | % {
    $localFiles = gci -Path $localPath # Get the local files
    # Change /content/foo/file.png --> file.png
    $file = $_.externalPath -replace "\/content\/$($contentLibraryName)\/"
    # Check to see if file exists locally
    $fileCheck = Get-ChildItem -Path "$($localPath)$($file)" -ErrorAction SilentlyContinue
    if(!$fileCheck) {
        # File in content library but not on disk; delete these files
        $null = Invoke-QlikDelete -path "/qrs/ContentLibrary/$($contentLibraryName)/deletecontent?externalpath=$($file)" -ErrorAction Continue
    } else {
        # File exists on disk and in content library
    }
    # foreach local file, upload and replace (if present)
    $localFiles | % {  
        $null = Import-QlikContent -LibraryName $contentLibraryName -FilePath "$($_.Name)" -Overwrite:$true -ErrorAction Continue
    }
}
    $contentLibraries = Get-QlikContentLibrary
    $contentLibrary = $contentLibraries | Where-Object {$_.Name -eq $contentLibraryName}
    $contentLibrary = Get-QlikContentLibrary -id $($contentLibrary.id)
    $contentLibraryFiles = $contentLibrary.references
    Write-Host "$($contentLibraryFiles.Count) files in Content Library"
    $i = $i + 1
    Start-Sleep -Seconds 90
    echo (Get-Date)
}