search

SneaksAndData / arcane-framework

Akka.NET-based framework for data streaming services using the Arcane Kubernetes Operator
Apache License 2.0
5 stars 2 forks source link

Add StreamConfigurationProvider #12

Closed s-vitaliy closed 6 months ago

s-vitaliy commented 7 months ago

Fixes/Implements #.

Scope

Implemented:

Additional changes:

Checklist

github-actions[bot] commented 7 months ago

Coverage after merging implement-stream-configuration-provider into main will be

57.38%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src/Metrics/Models
   SourceTags.cs0%100%0%0%13, 18, 23, 30–37
src/Services
   StreamConfigurationProvider.cs0%100%0%0%21–23
   StreamRunnerService.cs0%0%0%0%30–40, 44–49, 53–54, 54, 54–56, 59–61
src/Services/Base
   IStreamConfigurationProvider.cs0%100%0%0%11, 16
src/Services/Extensions
   StringExtensions.cs0%0%0%0%10, 17, 26, 26, 26–27
src/Sinks/Json
   JsonSink.cs74.66%42.86%100%77.68%102–105, 149–151, 151, 151, 151, 151, 151, 154–156, 158–159, 161–162, 164, 166, 177, 177–180, 82, 82, 82, 82, 84–86, 96–99
   MultilineJsonSink.cs79.12%52%100%81.69%113, 113, 113, 113, 115–117, 127–130, 133–136, 206–208, 208, 208, 208, 208, 208, 211–213, 215–216, 218–219, 221, 223, 234, 234–238
src/Sinks/Parquet
   ParquetSink.cs83.26%64.10%100%85.88%114, 114, 114, 114, 116–118, 134–137, 161, 212–215, 252–254, 254, 254, 254, 254, 254, 257–259, 261–262, 264–265, 267, 269, 280, 280, 286–288, 45
   ParquetOperations.cs44.16%18.75%85.71%53.54%100–105, 107–108, 148–150, 61–62, 62, 62–65, 65, 65–67, 69, 71–74, 74, 74–76, 76, 76–78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78–85, 85, 85, 85, 85, 85, 85, 85, 85–91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91–96, 98–99, 99, 99
src/Sinks/Parquet/Models
   DataCell.cs84.62%100%75%88.89%31
src/Sources/CdmChangeFeedSource
   CdmChangeFeedSource.cs0%0%0%0%114–115, 115, 115, 115, 115–118, 121, 121, 121, 121, 121–129, 133–135, 138–141, 143–144, 155, 159–163, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165–175, 178–179, 182, 185, 188–190, 190, 190–193, 195–198, 201–202, 202, 202, 206, 206, 206–209, 211–212, 214–215, 217, 220–223, 223, 223–224, 228–231, 233–236, 236, 236–238, 240, 240, 240–243, 247, 247, 247, 247, 247–252, 252, 252–257, 260–268, 270, 270, 270, 272–273, 275–276, 278–280, 283, 285–298, 301–348, 351–352, 352, 352, 352, 352–355, 357, 357, 357–359, 359, 359–363, 365–367, 369–371, 374–379, 379, 379, 379, 379–386, 386, 386–389, 391–395, 395, 395, 395, 395–400, 402–403, 406, 408–416, 46–64, 67, 72, 75, 79–81, 85–92
src/Sources/CdmChangeFeedSource/Extensions
   CsvOperations.cs90.22%97.22%33.33%88.68%54, 85–87, 95–97
   JsonDocumentOperations.cs0%100%0%0%18–24, 35–40, 49–53
   SimpleCdmAttributeExtensions.cs0%0%0%0%19–23, 23, 23, 23, 23–30, 30, 30, 30, 30–41, 41, 41–46, 49–53, 53, 53–55, 58–59
src/Sources/CdmChangeFeedSource/Models
   SimpleCdmAttribute.cs0%0%0%0%106–108, 108, 108–111, 111, 111–113, 116–118, 123–124, 124, 124–126, 129, 129, 129–131, 134, 134, 134–136, 139, 139, 139, 14, 140–141, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144–149, 15, 150, 153–156, 16–29, 35, 41, 47, 53, 59, 65, 70–71, 79–81, 90–91, 91, 91–93, 96–97
   SimpleCdmEntity.cs0%0%0%0%101, 101, 101–103, 106, 106, 106, 106, 106–109, 112–114, 18, 23, 28, 33–34, 42–56, 58, 60–61, 69–70, 72, 72, 72–75, 77, 79–80, 85–86, 86, src/Metrics/Models
   SourceTags.cs0%100%0%0%13, 18, 23, 30–37
src/Services
   StreamConfigurationManager.cs0%100%0%0%16–17, 19
   StreamRunnerService.cs0%0%0%0%30–40, 44–49, 53–54, 54, 54–56, 59–61
src/Services/Extensions
   StringExtensions.cs0%0%0%0%10, 17, 26, 26, 26–27
src/Sinks/Json
   JsonSink.cs74.66%42.86%100%77.68%102–105, 149–151, 151, 151, 151, 151, 151, 154–156, 158–159, 161–162, 164, 166, 177, 177–180, 82, 82, 82, 82, 84–86, 96–99
   MultilineJsonSink.cs79.12%52%100%81.69%113, 113, 113, 113, 115–117, 127–130, 133–136, 206–208, 208, 208, 208, 208, 208, 211–213, 215–216, 218–219, 221, 223, 234, 234–238
src/Sinks/Parquet
   ParquetSink.cs83.26%64.10%100%85.88%114, 114, 114, 114, 116–118, 134–137, 161, 212–215, 252–254, 254, 254, 254, 254, 254, 257–259, 261–262, 264–265, 267, 269, 280, 280, 286–288, 45
   ParquetOperations.cs44.16%18.75%85.71%53.54%100–105, 107–108, 148–150, 61–62, 62, 62–65, 65, 65–67, 69, 71–74, 74, 74–76, 76, 76–78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78–85, 85, 85, 85, 85, 85, 85, 85, 85–91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91–96, 98–99, 99, 99
src/Sinks/Parquet/Models
   DataCell.cs84.62%100%75%88.89%31
src/Sources/CdmChangeFeedSource
   CdmChangeFeedSource.cs0%0%0%0%114–115, 115, 115, 115, 115–118, 121, 121, 121, 121, 121–129, 133–135, 138–141, 143–144, 155, 159–163, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165–175, 178–179, 182, 185, 188–190, 190, 190–193, 195–198, 201–202, 202, 202, 206, 206, 206–209, 211–212, 214–215, 217, 220–223, 223, 223–224, 228–231, 233–236, 236, 236–238, 240, 240, 240–243, 247, 247, 247, 247, 247–252, 252, 252–257, 260–268, 270, 270, 270, 272–273, 275–276, 278–280, 283, 285–298, 301–348, 351–352, 352, 352, 352, 352–355, 357, 357, 357–359, 359, 359–363, 365–367, 369–371, 374–379, 379, 379, 379, 379–386, 386, 386–389, 391–395, 395, 395, 395, 395–400, 402–403, 406, 408–416, 46–64, 67, 72, 75, 79–81, 85–92
src/Sources/CdmChangeFeedSource/Extensions
   CsvOperations.cs90.22%97.22%33.33%88.68%54, 85–87, 95–97
   JsonDocumentOperations.cs0%100%0%0%18–24, 35–40, 49–53
   SimpleCdmAttributeExtensions.cs0%0%0%0%19–23, 23, 23, 23, 23–30, 30, 30, 30, 30–41, 41, 41–46, 49–53, 53, 53–55, 58–59
src/Sources/CdmChangeFeedSource/Models
   SimpleCdmAttribute.cs0%0%0%0%106–108, 108, 108–111, 111, 111–113, 116–118, 123–124, 124, 124–126, 129, 129, 129–131, 134, 134, 134–136, 139, 139, 139, 14, 140–141, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144–149, 15, 150, 153–156, 16–29, 35, 41, 47, 53, 59, 65, 70–71, 79–81, 90–91, 91, 91–93, 96–97
   SimpleCdmEntity.cs0%0%0%0%101, 101, 101–103, 106, 106, 106, 106, 106–109, 112–114, 18, 23, 28, 33–34, 42–56, 58, 60–61, 69–70, 72, 72, 72–75, 77, 79–80, 85–86, 86, 86–88, 91, 91, src/Metrics/Models
   SourceTags.cs0%100%0%0%13, 18, 23, 30–37
src/Services
   StreamConfigurationReader.cs0%100%0%0%16–17, 19
   StreamRunnerService.cs0%0%0%0%30–40, 44–49, 53–54, 54, 54–56, 59–61
src/Services/Extensions
   StringExtensions.cs0%0%0%0%10, 17, 26, 26, 26–27
src/Sinks/Json
   JsonSink.cs74.66%42.86%100%77.68%102–105, 149–151, 151, 151, 151, 151, 151, 154–156, 158–159, 161–162, 164, 166, 177, 177–180, 82, 82, 82, 82, 84–86, 96–99
   MultilineJsonSink.cs79.12%52%100%81.69%113, 113, 113, 113, 115–117, 127–130, 133–136, 206–208, 208, 208, 208, 208, 208, 211–213, 215–216, 218–219, 221, 223, 234, 234–238
src/Sinks/Parquet
   ParquetSink.cs83.26%64.10%100%85.88%114, 114, 114, 114, 116–118, 134–137, 161, 212–215, 252–254, 254, 254, 254, 254, 254, 257–259, 261–262, 264–265, 267, 269, 280, 280, 286–288, 45
   ParquetOperations.cs44.16%18.75%85.71%53.54%100–105, 107–108, 148–150, 61–62, 62, 62–65, 65, 65–67, 69, 71–74, 74, 74–76, 76, 76–78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78–85, 85, 85, 85, 85, 85, 85, 85, 85–91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91–96, 98–99, 99, 99
src/Sinks/Parquet/Models
   DataCell.cs84.62%100%75%88.89%31
src/Sources/CdmChangeFeedSource
   CdmChangeFeedSource.cs0%0%0%0%114–115, 115, 115, 115, 115–118, 121, 121, 121, 121, 121–129, 133–135, 138–141, 143–144, 155, 159–163, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165–175, 178–179, 182, 185, 188–190, 190, 190–193, 195–198, 201–202, 202, 202, 206, 206, 206–209, 211–212, 214–215, 217, 220–223, 223, 223–224, 228–231, 233–236, 236, 236–238, 240, 240, 240–243, 247, 247, 247, 247, 247–252, 252, 252–257, 260–268, 270, 270, 270, 272–273, 275–276, 278–280, 283, 285–298, 301–348, 351–352, 352, 352, 352, 352–355, 357, 357, 357–359, 359, 359–363, 365–367, 369–371, 374–379, 379, 379, 379, 379–386, 386, 386–389, 391–395, 395, 395, 395, 395–400, 402–403, 406, 408–416, 46–64, 67, 72, 75, 79–81, 85–92
src/Sources/CdmChangeFeedSource/Extensions
   CsvOperations.cs90.22%97.22%33.33%88.68%54, 85–87, 95–97
   JsonDocumentOperations.cs0%100%0%0%18–24, 35–40, 49–53
   SimpleCdmAttributeExtensions.cs0%0%0%0%19–23, 23, 23, 23, 23–30, 30, 30, 30, 30–41, 41, 41–46, 49–53, 53, 53–55, 58–59
src/Sources/CdmChangeFeedSource/Models
   SimpleCdmAttribute.cs0%0%0%0%106–108, 108, 108–111, 111, 111–113, 116–118, 123–124, 124, 124–126, 129, 129, 129–131, 134, 134, 134–136, 139, 139, 139, 14, 140–141, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144–149, 15, 150, 153–156, 16–29, 35, 41, 47, 53, 59, 65, 70–71, 79–81, 90–91, 91, 91–93, 96–97
   SimpleCdmEntity.cs0%0%0%0%101, 101, 101–103, 106, 106, 106, 106, 106–109, 112–114, 18, 23, 28, 33–34, 42–56, 58, 60–61, 69–70, 72, 72, 72–75, 77, 79–80, 85–86, 86, 86–88, 91, 91, src/Metrics/Models
   SourceTags.cs0%100%0%0%13, 18, 23, 30–37
src/Services
   StreamConfigurationReader.cs0%0%0%0%18–19, 21, 25–27, 27, 27–29
   StreamRunnerService.cs0%0%0%0%30–40, 44–49, 53–54, 54, 54–56, 59–61
src/Services/Extensions
   StringExtensions.cs0%0%0%0%10, 17, 26, 26, 26–27
src/Sinks/Json
   JsonSink.cs74.66%42.86%100%77.68%102–105, 149–151, 151, 151, 151, 151, 151, 154–156, 158–159, 161–162, 164, 166, 177, 177–180, 82, 82, 82, 82, 84–86, 96–99
   MultilineJsonSink.cs79.12%52%100%81.69%113, 113, 113, 113, 115–117, 127–130, 133–136, 206–208, 208, 208, 208, 208, 208, 211–213, 215–216, 218–219, 221, 223, 234, 234–238
src/Sinks/Parquet
   ParquetSink.cs83.26%64.10%100%85.88%114, 114, 114, 114, 116–118, 134–137, 161, 212–215, 252–254, 254, 254, 254, 254, 254, 257–259, 261–262, 264–265, 267, 269, 280, 280, 286–288, 45
   ParquetOperations.cs44.16%18.75%85.71%53.54%100–105, 107–108, 148–150, 61–62, 62, 62–65, 65, 65–67, 69, 71–74, 74, 74–76, 76, 76–78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78–85, 85, 85, 85, 85, 85, 85, 85, 85–91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91–96, 98–99, 99, 99
src/Sinks/Parquet/Models
   DataCell.cs84.62%100%75%88.89%31
src/Sources/CdmChangeFeedSource
   CdmChangeFeedSource.cs0%0%0%0%114–115, 115, 115, 115, 115–118, 121, 121, 121, 121, 121–129, 133–135, 138–141, 143–144, 155, 159–163, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165–175, 178–179, 182, 185, 188–190, 190, 190–193, 195–198, 201–202, 202, 202, 206, 206, 206–209, 211–212, 214–215, 217, 220–223, 223, 223–224, 228–231, 233–236, 236, 236–238, 240, 240, 240–243, 247, 247, 247, 247, 247–252, 252, 252–257, 260–268, 270, 270, 270, 272–273, 275–276, 278–280, 283, 285–298, 301–348, 351–352, 352, 352, 352, 352–355, 357, 357, 357–359, 359, 359–363, 365–367, 369–371, 374–379, 379, 379, 379, 379–386, 386, 386–389, 391–395, 395, 395, 395, 395–400, 402–403, 406, 408–416, 46–64, 67, 72, 75, 79–81, 85–92
src/Sources/CdmChangeFeedSource/Extensions
   CsvOperations.cs90.22%97.22%33.33%88.68%54, 85–87, 95–97
   JsonDocumentOperations.cs0%100%0%0%18–24, 35–40, 49–53
   SimpleCdmAttributeExtensions.cs0%0%0%0%19–23, 23, 23, 23, 23–30, 30, 30, 30, 30–41, 41, 41–46, 49–53, 53, 53–55, 58–59
src/Sources/CdmChangeFeedSource/Models
   SimpleCdmAttribute.cs0%0%0%0%106–108, 108, 108–111, 111, 111–113, 116–118, 123–124, 124, 124–126, 129, 129, 129–131, 134, 134, 134–136, 139, 139, 139, 14, 140–141, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144–149, 15, 150, 153–156, 16–29, 35, 41, 47, 53, 59, 65, 70–71, 79–81, 90–91, 91, 91–93, 96–97
   SimpleCdmEntity.cs0%0%0%0%101, 101, 101–103, 106, 106, 106, 106, 106–109, 112–114, 18, 23, 28, 33–34, 42–56, 58, 60–61, 69–70, 72, 72, 72–75, 77, 79–80, 85–86, 86, src/Metrics/Models
   SourceTags.cs0%100%0%0%13, 18, 23, 30–37
src/Services
   StreamConfigurationReader.cs0%0%0%0%19–20, 22, 26–28, 28, 28–30
   StreamRunnerService.cs0%0%0%0%30–40, 44–49, 53–54, 54, 54–56, 59–61
src/Services/Extensions
   StringExtensions.cs0%0%0%0%10, 17, 26, 26, 26–27
src/Sinks/Json
   JsonSink.cs74.66%42.86%100%77.68%102–105, 149–151, 151, 151, 151, 151, 151, 154–156, 158–159, 161–162, 164, 166, 177, 177–180, 82, 82, 82, 82, 84–86, 96–99
   MultilineJsonSink.cs79.12%52%100%81.69%113, 113, 113, 113, 115–117, 127–130, 133–136, 206–208, 208, 208, 208, 208, 208, 211–213, 215–216, 218–219, 221, 223, 234, 234–238
src/Sinks/Parquet
   ParquetSink.cs83.26%64.10%100%85.88%114, 114, 114, 114, 116–118, 134–137, 161, 212–215, 252–254, 254, 254, 254, 254, 254, 257–259, 261–262, 264–265, 267, 269, 280, 280, 286–288, 45
   ParquetOperations.cs44.16%18.75%85.71%53.54%100–105, 107–108, 148–150, 61–62, 62, 62–65, 65, 65–67, 69, 71–74, 74, 74–76, 76, 76–78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78–85, 85, 85, 85, 85, 85, 85, 85, 85–91, 91, 91, 91, 91, 91, 91, 91, 91, 91, 91–96, 98–99, 99, 99
src/Sinks/Parquet/Models
   DataCell.cs84.62%100%75%88.89%31
src/Sources/CdmChangeFeedSource
   CdmChangeFeedSource.cs0%0%0%0%114–115, 115, 115, 115, 115–118, 121, 121, 121, 121, 121–129, 133–135, 138–141, 143–144, 155, 159–163, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165–175, 178–179, 182, 185, 188–190, 190, 190–193, 195–198, 201–202, 202, 202, 206, 206, 206–209, 211–212, 214–215, 217, 220–223, 223, 223–224, 228–231, 233–236, 236, 236–238, 240, 240, 240–243, 247, 247, 247, 247, 247–252, 252, 252–257, 260–268, 270, 270, 270, 272–273, 275–276, 278–280, 283, 285–298, 301–348, 351–352, 352, 352, 352, 352–355, 357, 357, 357–359, 359, 359–363, 365–367, 369–371, 374–379, 379, 379, 379, 379–386, 386, 386–389, 391–395, 395, 395, 395, 395–400, 402–403, 406, 408–416, 46–64, 67, 72, 75, 79–81, 85–92
src/Sources/CdmChangeFeedSource/Extensions
   CsvOperations.cs90.22%97.22%33.33%88.68%54, 85–87, 95–97
   JsonDocumentOperations.cs0%100%0%0%18–24, 35–40, 49–53
   SimpleCdmAttributeExtensions.cs0%0%0%0%19–23, 23, 23, 23, 23–30, 30, 30, 30, 30–41, 41, 41–46, 49–53, 53, 53–55, 58–59
src/Sources/CdmChangeFeedSource/Models
   SimpleCdmAttribute.cs0%0%0%0%106–108, 108, 108–111, 111, 111–113, 116–118, 123–124, 124, 124–126, 129, 129, 129–131, 134, 134, 134–136, 139, 139, 139, 14, 140–141, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144–149, 15, 150, 153–156, 16–29, 35, 41, 47, 53, 59, 65, 70–71, 79–81, 90–91, 91, 91–93, 96–97
   SimpleCdmEntity.cs0%0%0%0%101, 101, 101–103, 106, 106, 106, 106, 106–109, 112–114, 18, 23, 28, 33–34, 42–56, 58, 60–61, 69–70, 72, 72, 72–75, 77, 79–80, 85–86, 86,