brendanhay / gogol

A comprehensive Google Services SDK for Haskell.
Other
282 stars 105 forks source link

Unable to execute a query in google big query #142

Closed karasarvani closed 2 weeks ago

karasarvani commented 5 years ago

I tried executing a query in BigQuery using this library gogol-bigquery. Please find the following code snippet, QueryRequest and QueryResponse generated. Please let me know why I am getting a Left instead of Right and is there anything missing in Request

JobsQuery :

JobsQuery' {_jqPayload = QueryRequest' {_qrLocation = Just "asia-south1", _qrUseQueryCache = True, _qrPreserveNulls = Nothing, _qrKind = "bigquery#queryRequest", _qrQueryParameters = Nothing, _qrQuery = Just "select order_id from projectId:dataset.tablename limit 100", _qrParameterMode = Nothing, _qrTimeoutMs = Nothing, _qrUseLegacySQL = True, _qrDryRun = Nothing, _qrMaxResults = Nothing, _qrDefaultDataSet = Nothing}, _jqProjectId = "projectId"}

Response:

Left ("Error in $: not enough input","")

Code:

txnreport :: ActionM () txnreport = do body <- jsonData :: ActionM HT.Body -- query <- liftIO $ constructQuery body let respFn = GT._cliResponse $ executeQuery "a" eitherVal <- liftIO $ runResourceT $ respFn mempty html $ T.pack $ show eitherVal

setParams :: QueryRequest -> QueryRequest setParams qr = qr & (qrLocation .~ (Just "asia-south1")) . (qrUseLegacySQL .~ True) . (qrQuery .~ (Just "select order_id from projectId:dataset.table limit 100"))

executeQuery :: T.Text -> GClient (Rs JobsQuery) executeQuery query = do let qr = queryRequest modified = setParams qr in requestClient $ jobsQuery modified "projectId"

mayank-17 commented 1 year ago

Here is how you use it

bqQueryRequest :: T.Text -> IO Value
bqQueryRequest query =  do
    bqProjectId <- getBqProjectId
    let
        queryRequest = GBT.newQueryRequest
            { connectionProperties = Nothing
            , createSession = Nothing
            , defaultDataset = Nothing
            , dryRun = Nothing
            , kind = "bigquery#queryRequest"
            , labels = Nothing
            , location = Just "asia-south1"
            , maxResults = Nothing
            , maximumBytesBilled = Nothing
            , parameterMode = Nothing
            , preserveNulls = Nothing
            , query = Just query
            , queryParameters = Nothing
            , requestId = Nothing
            , timeoutMs = Just 276447232
            , useLegacySql = False
            , useQueryCache = False
            }
        bigQueryJobsQuery = GBQ.BigQueryJobsQuery queryRequest (T.pack bqProjectId)

    lgr <- Google.newLogger Google.Error stdout
    _ <- print queryRequest
    manager <- newTlsManagerWith tlsManagerSettings
    googleEnv <-
            Google.newEnv
              <&>
                (Google.envLogger .~ lgr)
                . (Google.envScopes .~ (Proxy :: Proxy '[Bigquery'FullControl]))
                . (Google.envManager .~ manager)
    bqRes <- liftIO $ runResourceT $ Google.sendEither googleEnv mempty bigQueryJobsQuery

    case bqRes of
        Left _  -> print(("BQ Response Error " :: [Char]) <> show bqRes)
        Right _ -> print("Got BQ Response successful " :: [Char])
    return ( case bqRes of
                Right b -> toJSON b
                Left _  -> Null )
    where
        getBqProjectId :: IO String
        getBqProjectId = fromMaybe (error "BQ_PROJECT_ID not present in ENV") <$> (lookupEnv "BQ_PROJECT_ID")

Calling the function

bqQueryRequest "SELECT 1 AS a, 2 AS b FROM __table_name__"

tonicebrian commented 2 weeks ago

@karasarvani does @mayank-17 solved your question? Feel free to reopen if not.