Overview
Provider routing is a key feature of MythIQ that determines which AI service provider handles your request. Unlike model routing, which focuses on selecting the appropriate model, provider routing determines which implementation of a specific model to use when multiple providers offer the same model.
This system allows MythIQ to:
- Optimize for cost, performance, or feature compatibility
- Provide fallback options if a provider is unavailable
- Balance load across multiple providers
- Ensure high availability of the API
Intelligent Routing
MythIQ's provider routing system automatically selects the most appropriate provider based on your requirements, ensuring optimal performance, reliability, and cost-effectiveness.
Provider Selection Process
When you make a request to the MythIQ API, the system follows these steps to select a provider:
- Identify all providers that support the requested model
- Filter providers based on compatibility with your request parameters
- Apply your routing strategy to rank the remaining providers
- Select the highest-ranked provider
- If the selected provider fails, try the next provider in the ranking
Provider Selection Criteria
Providers are evaluated based on several factors:
Whether the provider supports all the parameters specified in your request
The price charged by the provider for the requested operation
Historical data on response times and success rates
Current status and recent uptime of the provider
Routing Strategies
MythIQ offers several routing strategies that determine how providers are selected. You can specify a strategy using the routing parameter:
optimal
(default)Balances cost and feature support. Selects the cheapest provider that supports all requested features, or may choose a slightly more expensive provider (within 20% price difference) if it supports more features.
Example
// Using the default 'optimal' strategy
const response = await openai.images.generate({
model: 'stability/sdxl',
prompt: 'A beautiful sunset over the ocean',
// routing: 'optimal' is implied
});cheapest
Always selects the lowest-cost provider that can fulfill the basic requirements of your request. This strategy prioritizes cost savings over additional features or capabilities.
Example
// Using the 'cheapest' strategy
const response = await openai.images.generate({
model: 'stability/sdxl',
prompt: 'A beautiful sunset over the ocean',
routing: 'cheapest',
});compatible
Selects a provider that supports all the parameters you've specified in your request, even if some are optional. This strategy ensures that all features you've requested are supported, even if it means using a more expensive provider.
Example
// Using the 'compatible' strategy with advanced parameters
const response = await openai.images.generate({
model: 'stability/sdxl',
prompt: 'A beautiful sunset over the ocean',
routing: 'compatible',
// These advanced parameters will ensure a provider that supports them is selected
cfg_scale: 10,
steps: 50,
});Specifying a Provider Explicitly
If you want to use a specific provider, you can bypass the routing system by specifying the provider parameter:
// Specifying a provider explicitly
const response = await openai.images.generate({
model: 'stability/sdxl',
prompt: 'A beautiful sunset over the ocean',
provider: 'stability', // Use Stability AI specifically
});Note on Explicit Provider Selection
When you specify a provider explicitly, the request will fail if that provider is unavailable or doesn't support the requested parameters. Consider using routing strategies for better reliability.
Provider Fallbacks
MythIQ automatically handles provider fallbacks when using routing strategies. If a provider fails to process your request (due to errors, timeouts, or other issues), the system will automatically try the next best provider according to your routing strategy.
Fallback Process
- Attempt the request with the primary provider
- If it fails, try the next provider in the ranking
- Continue until a provider succeeds or all options are exhausted
- Return the result from the successful provider or an error if all providers fail
The system will make up to 3 attempts across different providers before giving up. Each attempt is logged and can be seen in the response metadata.
Fallback Response Example
{
"data": [
{
"id": "abc123",
"url": "https://cdn.example.com/user/abc123/main.jpg",
"mime": "image/jpeg",
"model": "stability/sdxl",
"provider": "runpod", // Note: This might not be the first choice provider
"prompt": "A beautiful sunset over the ocean",
"size": "1024x1024"
}
],
"price": 0.002,
"success": true,
"stats": {
"setupTime": 120,
"processingTime": 350,
"generationTime": 2500,
"retries": 1, // Indicates a fallback was used
"providerId": "runpod"
}
}Provider Statistics
MythIQ collects and analyzes performance data from all providers to optimize the routing system. This data includes:
- Success rates
- Response times
- Error frequencies
- Cost variations
This information is used to continuously improve the routing algorithms and ensure the best possible experience for all users. The system adapts to changing conditions and provider performance over time.
Best Practices
Use Routing Strategies Instead of Explicit Providers
Whenever possible, use routing strategies rather than specifying providers explicitly. This provides better reliability and can optimize for your specific needs.
Choose the Right Strategy for Your Use Case
Select a routing strategy that aligns with your priorities:
optimalfor a balance of cost and featurescheapestfor cost-sensitive applicationscompatiblefor feature-rich requirements
Monitor Response Metadata
Pay attention to the response metadata, especially the stats field, which provides insights into how your request was processed and which provider was used.
Implement Proper Error Handling
Even with automatic fallbacks, requests may occasionally fail if all providers are unavailable or incompatible with your request. Implement proper error handling in your application to provide a good user experience in these rare cases.