Soheyla Mirshahi, Jan Kryštůfek – Beyond Accuracy: Engineering EU-Compliant LLM Systems

Soheyla Mirshahi, Jan Kryštůfek – Beyond Accuracy: Engineering EU-Compliant LLM Systems

Building production-ready LLM systems is challenging, but designing solutions that comply with the EU AI Act introduces layers of complexity far beyond prompt engineering and API integration. This talk takes you through the journey of creating a compliant LLM system for one of the most demanding domains: AI-driven recruitment. We share our journey from a naive PoC to a production ready solution under the AI Act. Through this case study, we will explore:

– Compliance-first approach: Designing LLM systems to meet security, legal and ethical standards from the ground up
– Taming LLM in Python: Using prompts, function calls and scoring model to get results in structured and auditable form
– Evaluation: Testing on synthetic data, human evaluation and user feedback
– Numbers: Statictics from production and percieved benefits

Attendees will leave equipped with practical tools, architectural patterns for building LLM systems that meet both engineering and regulatory challenges, supported by real-world example.

Presented at PyData Prague #24 – The Large, the Weed and the Compliant (3.12.2024 at Creative Dock)

www.pydata.org

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.

00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.

Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVideoTimestamps