This past Saturday was Boston Code Camp 39 (fall edition). For those of you who don’t know, the Boston Code Camp is the one that started it all (back in 2004) - see wikipedia for more info … at this point it may also be the last remaining one (not sure.).
I got to visit with many old friends that I mostly just see at Code Camp these days … many of which I met at a code camp some time in the past 20+ years. I did not get to attend any sessions due to taking the time to catch up with people instead.
I presented a “lessons learned” session, which was well attended and very interactive. I had a lot of really good questions regarding RAG, chunking, etc.
Lessons Learned from a Year of Building Copilot Agents & RAG Applications
The presentation was a split between Copilot Studio Agents and RAG Applications. Here are the lessons I discussed:
Copilot Studio Agents Lessons
- Simple RAG is Real Easy
- Observability is super simple
- (Copilot Studio) Agents that have conversation flow work the best
- Topics are powerful but may not be the solution you are looking for
- Agents tend to be specialized
RAG Applications Lessons
- Focus on the retrieval system
- Chunking strategies are not one size fits all
- Routing becomes more important as more features are added
- Scaling your knowledgebase will surface ingestion and retrieval problems
- Most development is just standard web applications with data and some prompt work
The full presentation pdf can be downloaded here
It was great to see some faces I hadn’t seen in awhile and nice to meet a bunch of new people.
If you have a comment, please message me @haleyjason on twitter/X.