Misadventures in AI: Notes from an A/E Firm

"So, what do you guys think of AI?"

I don't know if there's another question where my response changes with such high variability. Given how frequently we're being asked about it from contractors and clients, I figured it'd be worthwhile to document our journey to date, current thinking, and where we see things headed.

For posterity's sake, this article is being written on January 21st, 2026.

A Little About Myself

Let's disabuse ourselves of any notion that I am a competent data scientist or software engineer. I do have an analytical background, both formal training and a natural disposition, but my computer science capabilities max out at what we used to call a "script kiddie." I can read and write some Python, and somewhere I probably have a dusty drive with some embarrassing financial analysis scripts I will never share publicly.

What I do have is a broad understanding of programming languages and data structures, and a deep understanding of the Architecture and Engineering business. That puts me at an interesting crossroads, one where a lot of A/E firms find themselves right now.

The Only Framing You Need

• LLMs are really good at working with language and numbers: summarizing, drafting, classifying, organizing, and transforming.
• They are not magical. They do not "understand" in the way humans do. They can mimic reasoning and creation, but do not let them fool you.
• They are most useful in A/E when they are tightly constrained and grounded in authoritative sources. Don't trust, always verify.

That last point is the difference between a helpful assistant and a confident hallucination machine.

"The firms getting value from AI aren't the ones chasing the newest features. They're the ones doing the boring work: organizing information, structuring workflows, and setting constraints."

In other words, AI is best when you treat it like an extremely fast intern. Helpful, tireless, surprisingly capable, and absolutely capable of being wrong with extreme confidence.

Phase 1: GPT-4 as a Personal Tool

From March of '23 through fall 2025, our approach to AI remained largely unchanged. We used ChatGPT for things like cleaning up emails, simple research, and basic drafting. Useful, but mostly confined to individual productivity.

Phase 2: Firm Knowledge, Grounded in Our Standards

A few months after GPT-4 launched, we started exploring business use cases. We leveraged a slowdown in the multifamily business to hone our standards and build an intranet to make them more accessible. It became obvious that LLMs could make those resources far more usable, but only if we could ground the model in authoritative information and avoid it pulling random answers from the internet.

So we built an internal agent with Chatbase that could search a curated library of resources and respond in plain language. Two things became clear immediately:

• Knowledge has limits. These systems cannot just ingest "everything." File limits, size limits, and practical limits on retrieval quality forced us to split knowledge stores by use case.
• Stateless tools create friction. Our early tools had no memory. Session to session, they did not know who was talking or what had been discussed. That matters once you want consistency, personalization, or workflow continuity.

This phase gave us our first real A/E-oriented win: faster access to internal knowledge without relying on tribal memory or scavenger hunts through folders.

Phase 3: From Tools to Applications

Two things happened in the fall of 2025 that changed everything for us.

The first: OpenAI launched AgentKit: a suite of tools for building, deploying, and optimizing agents with visual workflow programming. For the first time, semi-competent laypeople like me could develop and deploy custom agents.

The second: Google launched Gemini 3. Its coding capabilities were far more robust than its predecessors, and this script kiddie was now dangerous. We could easily create and deploy Streamlit apps to test use cases. For the first time, the bottleneck wasn't technical skill; it was imagination.

This is what's become known as "vibe coding": describing what you want in plain language and letting the model generate the code. You don't need to know how to write it; you just need to know what you want it to do. You have never seen someone prompt so furiously.

The first and most evident use case was developing an agent to help staff respond to RFIs. At its heart, an RFI is a user feedback ticket. Something is wrong, and we need to fix it.

We strive to never repeat a mistake previously made. But how do we know if we previously made that mistake? If we could convert our unstructured RFI files into working RAG knowledge and connect it to external resources like building codes and standards, we could build an agent that helps staff:

• See how we have responded to similar RFIs in the past
• Validate those responses against the applicable regulatory framework
• Generate responses that meet baseline quality criteria (clarity, completeness, politeness)

Getting there required moving RFIs from local storage to cloud storage for processing. At some point over the Christmas holiday, I adopted Claude Code from Anthropic, which supercharged our progress. Once I moved to a workflow where the AI could debug and revise code in real time inside the IDE, we were off to the races.

Structuring the data turned out to be the crucial missing step we hadn't anticipated. RFIs need metadata. An RFI response from Texas in 1994 might not have meaningful application to a question being asked in Florida in 2025. We eventually settled on SQL. Our alpha testing is going quite well.

LLMs don't have a good grasp on drawing context, but there are a number of fairly easy-to-execute QC checks they excel at, especially when the goal is not to "design," but to reduce preventable errors and friction.

What we want automated support for:

• Cross-consultant coordination checks: obvious disconnects between disciplines that everyone "assumed someone else had"
• Life safety and common egress errors that hide in plain sight across large sets
• Consistency checks across sheets, keynote tags, schedules, and legends
• Accessibility items that show up as small misses but become big issues in the field
• Anything repetitive and rules-based, where fatigue is the real enemy

Humans are excellent at design intent, judgment calls, and understanding context across disciplines. We're also prone to fatigue, confirmation bias, and missing the "obvious" when we've looked at the same sheets for the third time at 11:30 PM. AI is great at repetitive checks, high-volume pattern matching, and consistency. It never gets tired, and it never assumes something is fine just because it looks familiar. The right balance is letting AI handle the grind while the professionals handle the responsibility.

We reviewed the marketplace and demoed several drawing review tools. One returned results in four minutes, with only 11 issues identified, on a set we knew had at least nine readily identifiable code violations, to say nothing of the 309 items we had flagged in our pre-IFC review. We then found InspectMind.AI. Their first pass through a 750-drawing multifamily project took two weeks, flagged most of the major issues we knew about, and surfaced 97 more we had missed. Even with a ~30% hit rate, the identified issues represent significant construction cost and schedule implications. The second project came back in two days with a notably improved UI. Progress is following an exponential path.

Unless you have a really great ERP, your current reporting and dashboard systems probably leave something wanting. We use Deltek on-prem and have had problems visualizing our financial data, but financial data is only one piece of the puzzle. What about quality tracking in Autodesk Construction Cloud? What about our internal SharePoint CA log?

With Claude Code, we've been able to map Deltek's SQL structure, connect to our SharePoint logs, and connect to the Autodesk APIs to generate a custom internal web server displaying fully customized dashboards in real time. A project manager can quickly glance and see key KPIs across their portfolio without hopping between platforms or chasing fragmented spreadsheets. Reporting becomes proactive instead of reactive.

We built a working internal prototype in about 11 hours. This would have easily cost over $20K with a contractor, and we'd have been left with limited customization options.

It took us a few weeks to find a model that could be tuned reliably to produce outputs that met our exacting requirements. Tuning is its own animal, but it can be tackled with proper effort.

Claude has made it extremely simple to build a lightweight app that takes an input (floor plan, elevation), processes it, makes an API call to the model, and returns a rendering ready to use. This is taking our rendering costs down from several thousand dollars to less than the price of a Starbucks coffee. Those are real savings we can pass on to clients, and it's letting us iterate and ideate like never before.

What's Next

The technology is moving fast. Our responsibility is to move thoughtfully.

As we evaluate and deploy these tools, we have to keep one thing front and center: these are tools. Tools do not replace professional judgment and reason. We cannot allow them to become a crutch, and we cannot skip training and development by delegating core competencies to software. This will require discipline and vigilance, and we plan to approach it with both.