Building with Google Gemini 3.1: Tools, Tips & What You Can Create

You saw the benchmark numbers. Gemini 3.1 scoring 77.1% on ARC-AGI-2. Leading 13 out of 16 major evaluations. And somewhere in that headline, your brain started turning over a half-formed idea you’ve been sitting on for months.

That’s the moment this post is for. Not the announcement. Not the marketing. The moment after — when you close the tab, open a new one, and start wondering: “Okay. What do I actually do with this? Where do I even start?”

This isn’t a benchmark breakdown or a feature comparison. It’s a practical guide to getting something real off the ground with Google’s latest models — what you can build, which tools to use, what actually works, and where things go wrong. We’ll be honest about the rough edges, because the gap between “works in a demo” and “works reliably for your users” is exactly where most AI projects stall.

Gemini 3 and 3.1: What Actually Changed

Gemini 3 launched in November 2025 as a significant generational leap. A 1-million-token context window that’s not just a spec number — it’s genuinely usable for ingesting entire codebases, research papers, or legal documents. Native multimodal understanding across text, images, audio, and video in a single model call. A “Deep Think” reasoning mode that improved accuracy substantially on complex tasks. And it sat at the top of the LMArena leaderboard for a while.

Then Gemini 3.1 arrived and addressed something Gemini 3 had been quietly criticized for: the tendency to rush toward a plausible-sounding answer rather than a correct one. Gemini 3.1 more than doubled performance on abstract reasoning tasks — that ARC-AGI-2 score is meaningful because that benchmark specifically tests reasoning that can’t be solved by memorizing training data. It leads 13 of 16 major evaluations now.

But the practical improvements for builders are just as important:

• File uploads up to 100MB — meaning you can feed the model entire datasets, PDFs, or codebases in one shot
• YouTube URL analysis — drop a link, get a full understanding of the video content without manual transcription
• Native SVG output — the model can write and animate SVG code directly with surprising accuracy
• Fixed the long-response truncation bug that cut off outputs mid-generation on complex tasks
• More reliable and consistent outputs — intentionally slower to “think through” before answering

Flash vs Pro: The Decision You’ll Make Every Time

Both Gemini 3 and 3.1 come in two variants, and picking the right one matters for your architecture and your cost model.

The practical rule: start everything with Flash. Once your logic is working, run a specific comparison on your hardest inputs. If Flash gets them right, you’re done. If it consistently falls short on complex multi-step tasks, that’s when Pro earns its cost.

The Google AI Ecosystem: Which Tool Do You Actually Use?

Google’s AI tooling has matured fast, and it’s easy to feel lost in the options. Here’s how to think about them clearly.

What You Can Actually Build with Gemini 3.1

Let’s be specific. Not “you can build AI apps” — what kinds of things actually become newly possible with these models that weren’t realistic six months ago?

Document Intelligence Tools

The 100MB upload limit combined with a 1-million-token context window means you can ingest a complete contract, a 300-page technical specification, or an entire codebase — and ask natural-language questions about it. Extract structured data from unstructured documents. Compare two versions of a legal agreement and highlight the changes. Summarize research papers with citations back to specific sections. This category of app was expensive and fragile six months ago. It’s genuinely accessible now.

Reasoning-Driven Agents

With thinking mode enabled and function calling connected to your external tools, you can build agents that don’t just retrieve information — they plan. An agent that takes a plain-English request like “find a time next week to meet with the Berlin team” and handles the calendar API calls, timezone math, and confirmation message without step-by-step instructions from you. The improved reasoning in 3.1 makes this category significantly more reliable — the model is better at deciding when to use a tool, what parameters to pass, and how to handle unexpected results.

Multimodal Applications

Gemini 3.1 processes text, images, audio, and video natively — not as separate pipeline stages, but as a unified understanding. A customer support tool where users upload a screenshot of the error they’re seeing, and the model explains what went wrong and how to fix it. A product review tool that accepts photos and generates detailed descriptions for your catalog. A meeting summarizer that takes the audio recording and outputs action items with owner assignments. The multimodal capability is real — it’s not just text-plus-image classification.

Long-Context Coding Assistants

One of the most underrated applications: give the model an entire codebase, ask it to find architectural issues, explain a module you’ve never touched, or write a new feature that integrates correctly with existing patterns. At 1 million tokens, you can fit a very large codebase into a single context. The model sees everything at once rather than guessing based on fragments. This is meaningfully different from the 32K or 128K window tools most developers have been working with.

YouTube-Powered Research Tools

Gemini 3.1 can analyze a YouTube URL directly — no downloading, no transcription service, no third-party API. Drop a link, get a summary, key timestamps, quotes, and insights. This makes it practical to build research tools, competitive intelligence dashboards, or content aggregators that pull signal from video content at scale. The kind of tool that previously required three different services can now be a single API call.

SVG and Creative Generation

This one surprised people. Gemini 3.1 can write SVG code with real accuracy — including animation. Icon generation, simple data visualizations, UI mockups, decorative elements. It’s not replacing a designer, but it’s a genuine creative accelerator for apps that need programmatic graphic output without the cost of a dedicated image generation API on every call.

What Actually Works: Tips for Building with Gemini 3.1

These are the things you learn by actually building — not from documentation.

1. Start with Flash, promote to Pro selectively

Don’t default to Pro because it sounds more capable. Build everything with Flash first. Flash is fast, significantly cheaper, and genuinely capable for most tasks. Once your core logic is working, run your hardest inputs through Flash and see where it struggles. Only upgrade specific calls to Pro when Flash consistently falls short on those cases. This keeps your costs predictable and your architecture honest.

2. Write system prompts like you’re onboarding a new employee

Vague system prompts produce vague outputs. Gemini 3.1 responds well to explicit role definition, specific rules, and concrete examples of what you do and don’t want. Don’t write “You are a helpful assistant.” Write “You are a contract review assistant for a SaaS company. Your job is to identify clauses that may create unexpected liability. You are not a lawyer. Always note when something requires professional legal review.” The specificity carries through to the output.

3. Use thinking mode intentionally — not by default

The thinking_level parameter controls how deeply the model reasons before responding. Higher thinking = slower response, more tokens, higher cost. It’s valuable for complex reasoning tasks where accuracy matters more than speed. It’s wasteful for simple extraction or classification tasks. Set it per task type, not globally across your app.

4. Pass thought signatures back in function calling — this is not optional

This one will catch you off guard if you don’t know about it. When using function calling with Gemini 3 models (including 3.1), the API returns thought signatures that must be passed back in subsequent turns. Skip this and you’ll get validation errors that aren’t obvious to debug. This applies even when using Flash with minimal thinking level. It’s not optional — it’s a validation requirement baked into the API.

response = client.models.generate_content(
    model="gemini-3.1-flash",
    contents=conversation_history,
    config=types.GenerateContentConfig(
        tools=[your_tools],
        thinking_config=types.ThinkingConfig(
            thinking_budget=1024
        )
    )
)

# Extract the thought signature from the response
thought_signature = response.candidates[0].content.parts[0].thought

# Pass it back in the next turn — this is required
conversation_history.append({
    "role": "model",
    "parts": [
        {"thought": True, "text": thought_signature},
        # ... rest of model response
    ]
})

5. Curate what goes in the context window

A 1-million-token context window is impressive. It doesn’t mean you should fill it. More irrelevant context introduces noise, increases cost, and can actually degrade response quality. Think of the context window as working memory, not storage. Put in what’s directly relevant to the task at hand. If you’re analyzing a specific contract clause, don’t also include the company’s entire file history unless it’s directly relevant.

6. Chain prompts rather than writing one massive instruction

Multi-step tasks are more reliable when broken into prompt chains — each step focused on one thing, feeding its output into the next. A single sprawling mega-prompt that asks the model to “analyze the document, identify key issues, cross-reference our policy, write a summary, and format it as a table” is harder to debug when something goes wrong. Break it apart. You get more predictable results, clearer failure modes, and a system that’s much easier to improve.

7. Model your API costs before you ship

The numbers look small until they don’t. At $2 per million input tokens, a feature that sends 50K tokens per request costs $0.10 per call. That’s manageable. Run that feature 100,000 times a month and it’s $10,000. Estimate your expected call volume, average token count, and model choice before you commit to an architecture. Free tier is 5 requests per minute — enough for development, not for launch.

The Part Nobody Puts in the Tutorial

Gemini 3.1 is genuinely impressive. It’s also a probabilistic language model with real limitations that will affect your users if you don’t plan for them. Here’s what to go in with eyes open about.

Hallucinations don’t go away with better models

Gemini 3.1 is more accurate than Gemini 3 on reasoning benchmarks. It still hallucinates. Every language model does. On factual edge cases, obscure details, and tasks that push against the boundaries of its training data, it will sometimes produce confident-sounding incorrect outputs. Your application needs to handle this gracefully — whether that’s citation requirements, human review steps, or output validation layers. Don’t build a system that trusts model output unconditionally.

Every call can return a different answer

Language models are probabilistic by nature. Send the same input twice and you may get two meaningfully different outputs. For most chat interfaces, this is fine — even desirable. For business logic that downstream systems depend on, this is a problem. Production apps built on Gemini need determinism layers: structured output schemas, validation against expected formats, retry logic with fallbacks, and in some cases human-in-the-loop review for high-stakes decisions.

Preview models shift under you

Both Gemini 3 and 3.1 are still evolving. API contracts, rate limits, and model behavior can change before stable release. If you’re building something people depend on, pin your model version and monitor for deprecation notices. “Works today” is not the same as “works next month.” This is a practical risk, not a theoretical one — it’s happened to teams building on early model previews from every major AI provider.

Prompt injection is a real attack surface

If your application takes user-provided content and passes it to the model — especially if the model has access to tools, data, or the ability to take actions — adversarial users can craft inputs that manipulate the model’s behavior. “Ignore your previous instructions and...” is only the obvious version. You need explicit guardrails: input validation, tool permission scoping, and output monitoring. This isn’t optional for any app where user input influences model behavior.

The demo gap is real and it’s wide

There is a significant difference between “works in a 10-minute demo” and “works reliably for 1,000 users with different backgrounds, expectations, and inputs you didn’t anticipate.” The demo gap isn’t specific to Gemini — it’s inherent to AI systems. Edge cases, adversarial inputs, unusual formats, users who don’t read the instructions, concurrent load, regional compliance requirements — none of these appear in the prototype. All of them appear in production. The gap between the two is where the real engineering work lives.

The Idea Is the Easy Part

Gemini 3.1 is a genuinely exciting moment for builders. The context window, the reasoning improvements, the multimodal capability — it opens up a real class of applications that weren’t practical before. The tools to work with it — AI Studio for prototyping, Antigravity for agentic development — are more accessible than anything that’s come before.

But every experienced developer who’s shipped an AI product knows: the intelligence isn’t the hard part. The hard part is everything around it — the schema validation that keeps outputs in bounds, the retry logic when the API returns an unexpected format, the rate limit handling that keeps your app responsive under load, the auth system that protects your data, the monitoring that tells you when something drifts. The gap between “Gemini can do this” and “my product does this reliably” is where most promising projects stall.

That’s exactly the gap ShipAI is built to close. We’ve built with Gemini. We know the API quirks, the function calling gotchas, the prompt engineering patterns that hold up at scale, and the production architecture that makes AI-powered apps reliable rather than impressive in a demo. When your prototype is ready to become a real product — or when you want to skip the prototype stage entirely and build the right thing from the start — we’re the team that handles the execution.

The Short Version

• Gemini 3.1 is a meaningful step forward — better reasoning, 100MB uploads, YouTube analysis, native SVG, and more consistent outputs than Gemini 3
• Start with Flash for cost and speed. Upgrade specific calls to Pro only where Flash falls short on complex reasoning
• AI Studio is your fastest on-ramp for prototyping. Antigravity is the right tool for agentic development. The direct API gives you full control for production
• The most interesting things to build: document intelligence, reasoning agents, multimodal tools, long-context coding assistants, and YouTube-powered research tools
• Don’t skip: thought signatures in function calling, context curation, prompt chaining, rate limit planning, and cost modeling
• The pitfalls are real: hallucinations, probabilistic outputs, preview instability, prompt injection, and the demo gap. None of them are dealbreakers — all of them require planning
• The gap from prototype to production is where the real work is. Build accordingly — or work with a team that knows how to close it