Hackathon Quickstart Guide

Fast-track guide for hackathon participants to build prediction market agents

Welcome Hackathon Participants!

This guide helps you quickly get started building prediction market agents. Whether you’re at a hackathon or just want a fast path to creating your first agent, this is your starting point.

What Are Prediction Markets?

Prediction markets allow people to trade on the outcome of future events. Participants buy and sell shares representing different outcomes, and prices reflect the crowd’s probability estimates.

Learn more: What are Prediction Markets?

Quick Setup

1. Install Dependencies

   Install the repository with Poetry (Python >=3.11):

   python3.11 -m pip install poetry
   python3.11 -m poetry install
   python3.11 -m poetry shell
   
2. Configure Environment

   Copy .env.example to .env and add your API keys:

   cp .env.example .env
   
3. Get API Keys

   Obtain the following API keys (most are free):

Required API Keys

GRAPH_API_KEY - Required

What it does: Query prediction market data

Get it from: https://thegraph.com (free)

Setup:

1. Create account at The Graph
2. Navigate to API Keys section
3. Create new API key
4. Add to .env: GRAPH_API_KEY=your_key_here

SERPER_API_KEY - Required

What it does: Google search functionality for research

Get it from: https://serper.dev (free tier available)

Setup:

1. Sign up at Serper.dev
2. Copy your API key from dashboard
3. Add to .env: SERPER_API_KEY=your_key_here

FIRECRAWL_API_KEY - Required

What it does: Web scraping for gathering evidence

Get it from: https://www.firecrawl.dev (free tier available)

Setup:

1. Create account at Firecrawl
2. Get API key from dashboard
3. Add to .env: FIRECRAWL_API_KEY=your_key_here

OPENAI_API_KEY - Required

What it does: LLM calls for making predictions

Get it from:

• For hackathon: Join Discord and ask organizers
• Otherwise: https://platform.openai.com

Setup: Add to .env: OPENAI_API_KEY=sk-...

BET_FROM_PRIVATE_KEY - Required

What it does: Wallet private key for placing bets on Gnosis Chain

Setup:

1. Install MetaMask
2. Create new wallet or use existing
3. Add Gnosis Chain network to MetaMask:
   • Click network selector (top left)
   • Click “Add a custom network”
   • Enter details:
     • Name: Gnosis Chain
     • RPC URL: https://rpc.gnosischain.com
     • Chain ID: 100
     • Symbol: XDAI
4. Export private key from MetaMask:
   • Click account menu
   • Account details > Export private key
5. Get xDai funds:
   • For hackathon: Ask organizers in Discord
   • Otherwise: Bridge from Ethereum mainnet
6. Add to .env: BET_FROM_PRIVATE_KEY=0x...

Never share your private key or commit it to version control!

MANIFOLD_API_KEY - For Benchmarking

What it does: Run benchmarks against Manifold markets

Get it from: https://manifold.markets (free)

Setup:

1. Create account at Manifold
2. Go to Settings > API
3. Generate new API key
4. Add to .env: MANIFOLD_API_KEY=your_key_here

Running Your First Agent

Once configured, run an existing agent to test your setup:

CoinFlip Agent (Simple)

python prediction_market_agent/run_agent.py coinflip omen

The Task

Goal: Implement new logic for trading on prediction markets that gets good predictions for cheap.

Success Criteria:

• Accuracy >50% (better than random)
• Cost-effective (considering API costs, LLM calls, etc.)
• Bonus points for creativity and novel approaches!

Recommended Steps

1. Study the Simplest Agent

   Look at prediction_market_agent/agents/coinflip_agent/deploy.py - this is the simplest possible agent:

   class DeployableCoinFlipAgent(DeployableTraderAgent):
       def verify_market(self, market_type: MarketType, market: AgentMarket) -> bool:
           return True
   
       def answer_binary_market(self, market: AgentMarket) -> ProbabilisticAnswer | None:
           decision = random.choice([True, False])
           return ProbabilisticAnswer(
               confidence=0.5,
               p_yes=Probability(float(decision)),
               reasoning="I flipped a coin to decide.",
           )
   

   Run it:

   python prediction_market_agent/run_agent.py coinflip omen
   

   Watch the logs to understand how agents work.

2. Study the Advanced Agent

   Examine prediction_market_agent/agents/advanced_agent/deploy.py - this agent actually retrieves information from the web:

   Key features:

   • Searches Google for relevant information
   • Scrapes content from top URLs
   • Uses LLM to analyze and predict

   Run it:

   python prediction_market_agent/run_agent.py advanced_agent omen
   

   This agent performs much better than CoinFlip because it uses real data.

3. Study a Top Performer

   Look at DeployablePredictionProphetGPT4oAgent - one of the best agents:

   Performance:

   • 60% success rate
   • $834.73 in profits
   • Top of the leaderboard

   Try it:

   • Test in Streamlit demo
   • See predictions and reasoning in real-time
   • Analyze on Dune Dashboard
4. Create Your Agent

   Modify the advanced agent or create your own:

   Ideas to try:

   • Change LLM prompts
   • Add more data sources (Twitter, news APIs, etc.)
   • Implement specialized logic for certain market types
   • Use different models (Claude, Gemini, etc.)
   • Add caching to reduce costs
   • Implement multi-step reasoning

   Remember: POCs with strong hypotheses are welcome if you don’t have time for full evaluation!

Example: Modifying the Advanced Agent

Here’s how to create your own agent based on AdvancedAgent:

prediction_market_agent/agents/your_agent/deploy.py

from prediction_market_agent_tooling.deploy.agent import DeployableTraderAgent
from prediction_market_agent_tooling.gtypes import Probability
from prediction_market_agent_tooling.markets.agent_market import AgentMarket
from prediction_market_agent_tooling.markets.data_models import ProbabilisticAnswer
from prediction_market_agent_tooling.tools.google_utils import search_google_serper
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel

from prediction_market_agent.tools.web_scrape.markdown import web_scrape
from prediction_market_agent.utils import APIKeys


class YourCustomAgent(DeployableTraderAgent):
    bet_on_n_markets_per_run = 3

    def answer_binary_market(self, market: AgentMarket) -> ProbabilisticAnswer | None:
        # 1. Search for information
        urls = search_google_serper(market.question)
        
        # 2. Filter and scrape
        contents = []
        for url in urls[:5]:
            if "manifold" not in url:  # Skip copy-paste results
                if scraped := web_scrape(url):
                    contents.append(scraped[:5000])  # Limit length
        
        if not contents:
            return None
        
        # 3. Your custom analysis logic here!
        # Example: Use a better prompt
        probability, confidence = self.analyze_with_llm(
            market.question, 
            contents
        )
        
        return ProbabilisticAnswer(
            confidence=confidence,
            p_yes=Probability(probability),
            reasoning=f"Analyzed {len(contents)} sources",
        )
    
    def analyze_with_llm(self, question: str, contents: list[str]) -> tuple[float, float]:
        agent = Agent(
            OpenAIModel(
                "gpt-4o-mini",
                provider=get_openai_provider(api_key=APIKeys().openai_api_key),
            ),
            system_prompt="You are an expert prediction market analyst.",
        )
        
        # Your custom prompt engineering here!
        result = agent.run_sync(
            f"""Question: {question}
            
Evidence from web:
{chr(10).join(contents)}

Analyze the evidence and provide:
1. Probability (0.0-1.0) that the answer is YES
2. Your confidence (0.0-1.0) in this prediction

Respond with two numbers separated by space."""
        ).output
        
        prob, conf = map(float, result.split())
        return prob, conf

Evaluation Methods

1. Run Benchmark (Fast)

Test against resolved Manifold markets:

python scripts/simple_benchmark.py --n 10

Before running, add your agent to the script:

scripts/simple_benchmark.py

from prediction_market_agent.agents.your_agent.deploy import YourCustomAgent

# In the main() function:
benchmarker = Benchmarker(
    markets=markets_deduplicated,
    agents=[
        BenchmarkAgent(agent=AdvancedAgent()),
        BenchmarkAgent(agent=YourCustomAgent()),  # Add here
    ],
    cache_path=cache_path,
    only_cached=only_cached,
)

This generates a markdown report comparing your agent’s accuracy to human traders.

2. Live Trading (Real evaluation)

Set bet_on_n_markets_per_run and run daily:

class YourCustomAgent(DeployableTraderAgent):
    bet_on_n_markets_per_run = 5  # Bet on 5 markets per run

By default, agents place tiny bets, so no worries about spending too much!

Note: ~10 new markets open daily on Omen, and existing ones resolve.

3. Manual Observation

Use the Streamlit app for interactive testing:

streamlit run src/app.py

Watch what your agent does for specific questions and iterate.

Need More API Keys?

If you want to use a 3rd party service that requires paid API keys, reach out to hackathon organizers in the Discord channel!

Running Multiple Agents

You can run multiple agents simultaneously to test different theories:

1. Create Separate Repos

   Clone the repository multiple times:

   git clone <repo-url> agent-1
   git clone <repo-url> agent-2
   
2. Separate Private Keys

   Each agent needs its own wallet/private key for tracking:

   • Create multiple MetaMask accounts
   • Use different private keys in each .env file
3. Deploy Each Agent

   Run each agent independently:

   # In agent-1 directory
   python prediction_market_agent/run_agent.py agent1 omen
   
   # In agent-2 directory
   python prediction_market_agent/run_agent.py agent2 omen
   

This lets you test multiple approaches in parallel on real markets!

Ideas to Explore

Better Prompts

Improve prompt engineering for more accurate predictions

More Data Sources

Add Twitter, Reddit, news APIs for better context

Specialized Agents

Focus on specific categories (crypto, politics, sports)

Multi-Model Ensemble

Combine predictions from multiple LLMs

Historical Analysis

Learn from past market outcomes

Cost Optimization

Reduce API costs while maintaining accuracy

Chain-of-Thought

Implement step-by-step reasoning

Market Timing

Only trade when you have strong signals

Production Deployment

If your agent achieves at least 50% accuracy, it may be added to production deployment!

Benefits:

• Your agent runs live at Presagio
• Appears on the leaderboard
• Tracked on Dune Dashboard
• Trades on real markets 24/7

Requirements:

• 50% accuracy (proxy for not losing money)

• Your approval to use the agent
• Clean, maintainable code

Resources

GitHub Repository

View source code and issues

Discord Community

Get help and share progress

Presagio Leaderboard

See top-performing agents

Dune Dashboard

Track on-chain activity

Streamlit Demo

Interactive agent testing

Tips for Success

Start Simple

Begin with the CoinFlip or Advanced agent, make small changes, and test frequently.

Focus on Evidence

The best agents gather high-quality evidence before making predictions.

Manage Costs

Track API costs (OpenAI, Tavily, etc.) - they can add up quickly!

Iterate Quickly

Use benchmarking for fast iteration, then validate on live markets.

Read the Code

The best learning comes from reading existing agents’ code.

Ask for Help

Join Discord and ask questions - the community is helpful!

Common Issues

ModuleNotFoundError

Solution: Activate Poetry shell:

poetry shell

API Key Errors

Solution: Check your .env file:

• Ensure all required keys are present
• No spaces around = signs
• Keys are valid and have credits

No Markets Found

Solution:

• Check GRAPH_API_KEY is valid
• Verify network connectivity
• Try different market type (manifold, omen)

Transaction Failures

Solution:

• Ensure wallet has xDai balance
• Check private key is correct format (starts with 0x)
• Verify you’re on Gnosis Chain (chain ID 100)

Next Steps

Create Your Agent

Detailed guide on building custom agents

Benchmark Your Agent

Test accuracy against human traders

Deploy to Production

Take your agent live

Good Luck!

Build something awesome, have fun, and may your predictions be ever accurate! 🚀