2991 remove SFTP and ACF titan related codes

Summary of Changes

Pull request closes #2991 _

PR removes all logic for transferring uploaded files to ACF TITAN via SFTP

How to Test

List the steps to test the PR These steps are generic, please adjust as necessary.

cd tdrs-frontend && docker-compose -f docker-compose.yml -f docker-compose.local.yml up -d
cd tdrs-backend && docker-compose -f docker-compose.yml -f docker-compose.local.yml up -d

Open http://localhost:3000/ and sign in.
Proceed with functional tests as described herein.
Test steps should be captured in the demo GIF(s) and/or screenshots below.

Demo GIF(s) and screenshots for testing procedure

Screenshot 2024-05-31 at 11 41 37 AM

Deliverables

More details on how deliverables herein are assessed included here.

Deliverable 1: Accepted Features

Checklist of ACs:

[x] files submitted through TDP are not transferred to ACFTitan server.

sftpp2 sftpp1

[ ] file transfer env vars are no longer present in the codebase, circleci, or backend environments
[x] backend logs do not show evidence of file transfer attempts.

 2024-06-06T12:42:32.28-0400 [APP/PROC/WEB/0] OUT 10.255.141.85 - - [06/Jun/2024:16:42:32 +0000] "GET /v1/stts/alpha HTTP/1.0" 200 36455 "https://tdp-frontend-qasp.app.cloud.gov/data-files" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
   2024-06-06T12:42:34.58-0400 [APP/PROC/WEB/0] ERR [2024-06-06 16:42:34 +0000] [569] [DEBUG] GET /v1/data_files/
   2024-06-06T12:42:34.63-0400 [APP/PROC/WEB/0] OUT 10.255.141.85 - - [06/Jun/2024:16:42:34 +0000] "GET /v1/data_files/?year=2024&quarter=Q1&stt=1&file_type=tanf HTTP/1.0" 200 740 "https://tdp-frontend-qasp.app.cloud.gov/data-files" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
   2024-06-06T12:42:37.47-0400 [APP/PROC/WEB/0] ERR [2024-06-06 16:42:37 +0000] [566] [DEBUG] Closing connection.
   2024-06-06T12:42:38.26-0400 [APP/PROC/WEB/0] ERR [2024-06-06 16:42:38 +0000] [568] [DEBUG] POST /v1/data_files/
   2024-06-06T12:42:38.30-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,309 DEBUG views.py::create:L57 :  DataFileViewSet: <rest_framework.request.Request: POST '/v1/data_files/'>
   2024-06-06T12:42:38.30-0400 [APP/PROC/WEB/0] ERR DataFileViewSet: <rest_framework.request.Request: POST '/v1/data_files/'>
   2024-06-06T12:42:38.31-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,310 DEBUG validators.py::validate_file_infection:L54 :  CLAMAV_NEEDED noted as True, proceeding with scan.
   2024-06-06T12:42:38.31-0400 [APP/PROC/WEB/0] ERR CLAMAV_NEEDED noted as True, proceeding with scan.
   2024-06-06T12:42:38.31-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,310 DEBUG clients.py::__init__:L37 :  Set clamav endpoint_url as 'http://tdp-clamav-nginx-dev.apps.internal:9000/scan'
   2024-06-06T12:42:38.31-0400 [APP/PROC/WEB/0] ERR Set clamav endpoint_url as 'http://tdp-clamav-nginx-dev.apps.internal:9000/scan'
   2024-06-06T12:42:38.31-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,311 DEBUG clients.py::scan_file:L64 :  Initiating virus scan for file: test_staging.txt
   2024-06-06T12:42:38.31-0400 [APP/PROC/WEB/0] ERR Initiating virus scan for file: test_staging.txt
   2024-06-06T12:42:38.32-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,323 DEBUG clients.py::scan_file:L90 :  File scan marked as CLEAN for file: test_staging.txt
   2024-06-06T12:42:38.32-0400 [APP/PROC/WEB/0] ERR File scan marked as CLEAN for file: test_staging.txt
   2024-06-06T12:42:38.61-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,613 DEBUG views.py::create:L64 :  DataFileViewSet: status: 201
   2024-06-06T12:42:38.61-0400 [APP/PROC/WEB/0] ERR DataFileViewSet: status: 201
   2024-06-06T12:42:38.61-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,617 INFO views.py::create:L69 :  Preparing parse task: User META -> user: alexandra.pennington@acf.hhs.gov, stt: Alabama (01). Datafile META -> datafile: 642, section: Active Case Data, quarter Q1, year 2024.
   2024-06-06T12:42:38.61-0400 [APP/PROC/WEB/0] ERR Preparing parse task: User META -> user: alexandra.pennington@acf.hhs.gov, stt: Alabama (01). Datafile META -> datafile: 642, section: Active Case Data, quarter Q1, year 2024.
   2024-06-06T12:42:38.65-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,655 INFO views.py::create:L74 :  Submitted parse task to queue for datafile 642.
   2024-06-06T12:42:38.65-0400 [APP/PROC/WEB/0] ERR Submitted parse task to queue for datafile 642.
   2024-06-06T12:42:38.68-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,689 INFO parser_task.py::parse:L25 :  DataFile parsing started for file ADS.E2J.NDM1.TS01
   2024-06-06T12:42:38.68-0400 [APP/PROC/WEB/0] ERR [2024-06-06 16:42:38,689: INFO/ForkPoolWorker-1] DataFile parsing started for file ADS.E2J.NDM1.TS01
   2024-06-06T12:42:38.82-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,822 DEBUG views.py::create:L83 :  DataFileViewSet: return val: <Response status_code=201, "text/html; charset=utf-8">
   2024-06-06T12:42:38.82-0400 [APP/PROC/WEB/0] ERR DataFileViewSet: return val: <Response status_code=201, "text/html; charset=utf-8">
   2024-06-06T12:42:38.82-0400 [APP/PROC/WEB/0] OUT 10.255.141.85 - - [06/Jun/2024:16:42:38 +0000] "POST /v1/data_files/ HTTP/1.0" 201 430 "https://tdp-frontend-qasp.app.cloud.gov/data-files" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"
   2024-06-06T12:42:38.83-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,834 INFO row_schema.py::parse_and_validate:L60 :  2 preparser error(s) encountered.
   2024-06-06T12:42:38.83-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,834 INFO parse.py::parse_datafile:L34 :  Preparser Error: 2 header errors encountered.
   2024-06-06T12:42:38.83-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,834 DEBUG parse.py::bulk_create_errors:L150 :  Bulk creating ParserErrors.
   2024-06-06T12:42:38.83-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,839 INFO parse.py::bulk_create_errors:L153 :  Created 2/1 ParserErrors.
   2024-06-06T12:42:38.85-0400 [APP/PROC/WEB/0] ERR 2024-06-06 16:42:38,850 INFO parser_task.py::parse:L38 :  Parsing finished for file -> {id: 642, filename: test_staging.txt, STT: Alabama (01), S3 location: data_files/2024/Q1/1/Active Case Data/test_staging.txt} with status Rejected and 1 errors.
   2024-06-06T12:42:38.85-0400 [APP/PROC/WEB/0] ERR [2024-06-06 16:42:38,850: INFO/ForkPoolWorker-1] Parsing finished for file -> {id: 642, filename: test_staging.txt, STT: Alabama (01), S3 location: data_files/2024/Q1/1/Active Case Data/test_staging.txt} with status Rejected and 1 errors.
   2024-06-06T12:42:38.92-0400 [APP/PROC/WEB/0] ERR [2024-06-06 16:42:38 +0000] [567] [DEBUG] POST /v1/logs/
   2024-06-06T12:42:38.96-0400 [APP/PROC/WEB/0] OUT 10.255.141.85 - - [06/Jun/2024:16:42:38 +0000] "POST /v1/logs/ HTTP/1.0" 200 9 "https://tdp-frontend-qasp.app.clo

[x] all other existing TDP features are functioning as expected.
[x] Testing Checklist has been run and all tests pass
[ ] README is updated, if necessary
[x] lfrohlich and/or adpennington confirmed that ACs are met.

Deliverable 2: Tested Code

Are all areas of code introduced in this PR meaningfully tested?
- [ ] If this PR introduces backend code changes, are they meaningfully tested?
- [ ] If this PR introduces frontend code changes, are they meaningfully tested?
Are code coverage minimums met?
- [ ] Frontend coverage: [insert coverage %] (see CodeCov Report comment in PR)
- [x] Backend coverage: [insert coverage %] (see CodeCov Report comment in PR)

Deliverable 3: Properly Styled Code

[x] Are backend code style checks passing on CircleCI?
[ ] Are frontend code style checks passing on CircleCI?
[ ] Are code maintainability principles being followed?

Deliverable 4: Accessible

[ ] Does this PR complete the epic?
[ ] Are links included to any other gov-approved PRs associated with epic?
[ ] Does PR include documentation for Raft's a11y review?
[ ] Did automated and manual testing with iamjolly and ttran-hub using Accessibility Insights reveal any errors introduced in this PR?

Deliverable 5: Deployed

[x] Was the code successfully deployed via automated CircleCI process to development on Cloud.gov?

Deliverable 6: Documented

[x] Does this PR provide background for why coding decisions were made?
[ ] If this PR introduces backend code, is that code easy to understand and sufficiently documented, both inline and overall?
[ ] If this PR introduces frontend code, is that code easy to understand and sufficiently documented, both inline and overall?
[ ] If this PR introduces dependencies, are their licenses documented?
[x] Can reviewer explain and take ownership of these elements presented in this code review?

Deliverable 7: Secure

[x] Does the OWASP Scan pass on CircleCI?
[ ] Do manual code review and manual testing detect any new security issues?
[ ] If new issues detected, is investigation and/or remediation plan documented?

Deliverable 8: User Research

Research product(s) clearly articulate(s):

[ ] the purpose of the research
[ ] methods used to conduct the research
[ ] who participated in the research
[ ] what was tested and how
[ ] impact of research on TDP
[ ] (if applicable) final design mockups produced for TDP development

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 93.10%. Comparing base (00ac3d1) to head (4e92ee0). Report is 1 commits behind head on develop.

Additional details and impacted files

[![Impacted file tree graph](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003/graphs/tree.svg?width=650&height=150&src=pr&token=BA04YXPAL9&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech)](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech) ```diff @@ Coverage Diff @@ ## develop #3003 +/- ## =========================================== + Coverage 93.05% 93.10% +0.05% =========================================== Files 276 275 -1 Lines 7108 7036 -72 Branches 599 596 -3 =========================================== - Hits 6614 6551 -63 + Misses 399 391 -8 + Partials 95 94 -1 ``` | [Flag](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech) | Coverage Δ | | |---|---|---| | [dev-backend](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech) | `93.18% <100.00%> (+0.06%)` | :arrow_up: | | [dev-frontend](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech) | `92.62% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech) | Coverage Δ | | |---|---|---| | [tdrs-backend/tdpservice/data\_files/views.py](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003?src=pr&el=tree&filepath=tdrs-backend%2Ftdpservice%2Fdata_files%2Fviews.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech#diff-dGRycy1iYWNrZW5kL3RkcHNlcnZpY2UvZGF0YV9maWxlcy92aWV3cy5weQ==) | `88.99% <100.00%> (-0.20%)` | :arrow_down: | | [tdrs-backend/tdpservice/settings/common.py](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003?src=pr&el=tree&filepath=tdrs-backend%2Ftdpservice%2Fsettings%2Fcommon.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech#diff-dGRycy1iYWNrZW5kL3RkcHNlcnZpY2Uvc2V0dGluZ3MvY29tbW9uLnB5) | `99.29% <ø> (-0.03%)` | :arrow_down: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech) ------ [Continue to review full report in Codecov by Sentry](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003?dropdown=coverage&src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech). Last update [cc37558...4e92ee0](https://app.codecov.io/gh/raft-tech/TANF-app/pull/3003?dropdown=coverage&src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=raft-tech).

raft-tech / TANF-app