The Latent Space

Recent Notes

Initial Front End Plan

4 min read

Your new approach is a great idea for a prototype! Keeping the UI simple and functional while providing the ability to select multiple providers in a dropdown is the perfect balance between usability and flexibility. Here’s why I think it’s a solid choice and some suggestions to make it even better:

Why This Approach Works for a Prototype

  1. Simplicity:

    • A single window with a dropdown keeps the UI clean and easy to understand for first-time users.
    • It eliminates the complexity of managing multiple dynamic chat windows but still offers multi-provider functionality.

  2. Core Functionality:

    • You focus on ensuring the backend handles requests from multiple providers correctly and efficiently.
    • Users can still compare responses from multiple LLMs, which is one of the primary goals.

  3. Easier Expansion Later:

    • Once the prototype is complete and stable, you can easily evolve it into a multi-window system or more advanced UX without needing a complete overhaul.

Suggested User Flow

  1. Initial UI:

    • A single input field for the query at the bottom.
    • A dropdown menu at the top labeled “Chat Providers” where users can select one or more LLM providers.
    • A “Submit” button to send the query to the selected providers.

  2. After Submission:

    • Display the user’s query at the top of the chat history.
    • Show each provider’s response below, labeled with the provider’s name (e.g., “ChatGPT: [response]”).
    • Optionally, include a timestamp or other metadata.

Implementation Steps

1. Frontend

  • Dropdown for Providers:

    • Use a multi-select dropdown (e.g., in React, you can use libraries like react-select with multi-select enabled).
    • Each item in the dropdown corresponds to a provider (ChatGPT, Grok, Gemini, etc.).

  • Query Submission:

    • When the user submits a query, collect:
      • The selected providers.
      • The input query.
      • Any other optional settings (e.g., temperature, max tokens).

  • Response Display:

    • Display responses grouped by provider in a scrollable chat history format.

2. Backend

  • Update Endpoint (/query):

    • Add support for receiving a list of providers as a query parameter (e.g., providers=chatgpt,grok).
    • Modify the ProviderFactory logic to only instantiate the selected providers.

  • Updated Endpoint Logic:

    @router.post("/")
async def query_api(
    request: QueryRequest,
    providers: Optional[List[str]] = Query(None),  # Selected providers
    factory: ProviderFactory = Depends(get_provider_factory),
):
    """
    Query selected providers and return their responses.
    """
    # Create providers based on user selection
    all_providers = factory.create_providers()
    selected_providers = [
        provider for provider in all_providers
        if provider.name.lower() in (providers or [p.name.lower() for p in all_providers])
    ]
 
    # Run queries concurrently
    tasks = [
        provider.send_query(request.prompt)
        for provider in selected_providers
    ]
    results = await asyncio.gather(*tasks, return_exceptions=True)
 
    # Collect responses
    responses = {
        provider.name: (
            result if not isinstance(result, Exception)
            else f"Error: {str(result)}"
        )
        for provider, result in zip(selected_providers, results)
    }
 
    return {"prompt": request.prompt, "responses": responses}

3. Updated Config

  • You don’t need to change your existing config file for this, but you might consider adding some default providers to be selected when the dropdown is empty.

Suggestions for Later Expansion

  • Selectable Settings:
    • Allow users to configure optional parameters like temperature or max tokens per provider in the UI.
  • Response Analysis:
    • Add a summary or comparison of responses from different providers in the UI (e.g., a highlight of differences).
  • Persist User Choices:
    • Use browser storage or a lightweight database to save the user’s last-used providers for convenience.

Advantages of This Approach

  1. Rapid Development:
    • You can implement this UI without heavy changes to the backend.
  2. Core Features Delivered:
    • Users get the ability to query multiple providers and view responses side by side.
  3. Scalable Design:
    • This setup lays the groundwork for more complex features (like multiple chat windows) in the future.

Potential Challenges

  • Multi-provider Overhead:
    • Sending queries to multiple providers might cause delays if a provider is slow to respond. You could mitigate this by setting a timeout for each provider.
  • Error Handling:
    • Ensure robust error handling in the backend so that one failing provider doesn’t crash the entire request.

Final Thoughts

This streamlined approach gives you a strong prototype while keeping things manageable. The single window with a multi-select dropdown aligns well with your backend’s current architecture and provides an excellent foundation for a polished MVP. Let me know if you need help implementing any specific part!

Graph View

Backlinks