Open rgarfield11 opened 8 months ago
Based on the code and context provided, the Text Chunker is a library written in Elixir, designed for splitting text into chunks based on certain parameters such as chunk size, overlap, and format. To create an interactive testing environment for this on a website, we would need to do the following:
Chunker.TextChunker
module to process the text.Since Elixir is not a common language for frontend development, we'd likely use JavaScript for the frontend. Our Elixir application would provide an API endpoint to process the chunking requests.
Here is an example of how the Elixir backend might provide an API endpoint using Phoenix Framework, assuming this application grows to include Phoenix:
# First, add Phoenix to deps in mix.exs
defp deps do
[
# ... existing deps
{:phoenix, "~> 1.5.7"},
]
end
# lib/text_chunker_web/controllers/chunk_controller.ex
defmodule TextChunkerWeb.ChunkController do
use TextChunkerWeb, :controller
alias Chunker.TextChunker
def create(conn, %{"text" => text, "options" => options}) do
# Parse chunking options and pass them to the TextChunker
opts = parse_options(options)
chunked_data = TextChunker.split(text, opts)
conn
|> put_status(:ok)
|> json(chunked_data)
end
defp parse_options(options) do
# Options parsing logic here
# Return options in the proper keyword list format expected by TextChunker.split/2
# Note: Be cautious of user input and validate/sanitize accordingly
# e.g., converting string values to integers
end
end
# lib/text_chunker_web/router.ex
defmodule TextChunkerWeb.Router do
use TextChunkerWeb, :router
pipeline :api do
plug :accepts, ["json"]
end
scope "/api", TextChunkerWeb do
pipe_through :api
post "/chunks", ChunkController, :create
end
end
For the frontend, you could use any modern framework such as React, Vue.js, or Angular. However, here is a simple HTML format with JavaScript (using the Fetch API) for interaction:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Text Chunker</title>
</head>
<body>
<h1>Text Chunker</h1>
<textarea id="textInput" rows="10" cols="50" placeholder="Enter your text here"></textarea>
<br>
<input type="number" id="chunkSize" placeholder="Chunk Size">
<input type="number" id="chunkOverlap" placeholder="Chunk Overlap">
<button id="chunkButton">Chunk Text</button>
<div id="chunkOutput"></div>
<script>
document.getElementById('chunkButton').addEventListener('click', function() {
const text = document.getElementById('textInput').value;
const chunkSize = document.getElementById('chunkSize').value;
const chunkOverlap = document.getElementById('chunkOverlap').value;
fetch('/api/chunks', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
text: text,
options: {
chunk_size: parseInt(chunkSize, 10),
chunk_overlap: parseInt(chunkOverlap, 10)
}
})
})
.then(response => response.json())
.then(data => {
document.getElementById('chunkOutput').innerText = JSON.stringify(data, null, 2);
})
.catch((error) => {
console.error('Error:', error);
});
});
</script>
</body>
</html>
This code covers most of the acceptance criteria. To ensure the chunking is done within a reasonable response time, you may need to optimize the backend code, possibly with caching or improving the chunking algorithm's efficiency for large documents if necessary.
Lastly, before deploying this interactive testing environment to production, thorough testing must be done to ensure reliability and security of the processing, especially with user-provided input data.
created by stuart.page+demos@revelry.co using Prodops
Background
This library is written; but it would be really cool if a website existed where users could test it out first.
Acceptance Criteria
Scenario: Users can perform text chunking on the website
Given I am a user on the Text Chunker Website
[ ] And the chunking process should be done within a reasonable response time
created by stuart.page+demos@revelry.co using Prodops