Job Title
Senior AI Engineer – LLM, RAG & Agentic AI
Location
Tokyo, Japan
Workplace Type
Hybrid / Flexible
Employment Type
Full-time
Company Overview
A rapidly growing global AI software company is transforming professional workflows through advanced generative AI solutions. Operating across Japan, North America, and Europe, the organization develops AI-powered platforms that help professionals automate complex decision-making and knowledge-intensive work. Backed by leading global investors and strategic technology partners, the company continues to expand its international footprint while driving innovation in AI-native enterprise software.
Position Overview
The Senior AI Engineer will lead the design, development, and operation of next-generation AI applications powered by Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and AI agents.
This role extends beyond model implementation and focuses on building production-grade AI systems that integrate seamlessly into real business workflows. The successful candidate will work closely with product, design, and engineering teams to create reliable, scalable, and user-centric AI experiences while driving technical decisions across the entire AI application lifecycle.
Key Responsibilities
AI Product Development
Design, develop, and operate AI-native product capabilities leveraging LLMs, RAG architectures, and AI agents.
Build highly reliable agent-based workflows for production environments.
Integrate AI capabilities into customer-facing products and business processes.
Collaborate with product and design teams to create intuitive AI-powered user experiences.
Translate business requirements into scalable AI solutions.
LLM & Agent Architecture
Lead architecture decisions related to model selection, inference strategies, retrieval frameworks, and agent orchestration.
Design and implement multi-step agent workflows using modern AI agent frameworks.
Evaluate emerging AI models, tools, and frameworks to identify optimal solutions for specific use cases.
Optimize model performance, latency, scalability, and operational efficiency.
Retrieval-Augmented Generation (RAG)
Design and implement advanced RAG architectures.
Develop retrieval pipelines utilizing embeddings, vector databases, and semantic search technologies.
Improve retrieval quality and knowledge grounding mechanisms.
Optimize context management and prompt engineering strategies.
Enhance answer accuracy and relevance through evaluation-driven improvements.
AI Platform Operations & Reliability
Build monitoring, logging, evaluation, and observability frameworks for AI applications.
Establish performance measurement methodologies and quality assurance standards.
Implement cost monitoring and optimization strategies for AI workloads.
Develop operational processes to ensure reliability, scalability, and maintainability.
Drive continuous improvement through experimentation and performance analysis.
Cross-Functional Collaboration
Work directly with stakeholders to gather requirements and define AI use cases.
Collaborate with engineering, product, and business teams to deliver AI solutions.
Contribute to technical standards, code reviews, and engineering best practices.
Create technical documentation and knowledge-sharing resources.
Mentor engineers and support technical decision-making across projects.
Required Qualifications
5+ years of software development experience using one or more of:
Python
TypeScript
Go
Hands-on experience building applications utilizing modern LLM APIs, including:
OpenAI
Anthropic
Google Gemini
Mistral
Similar foundation models
Experience developing AI agents using frameworks such as:
LangChain
CrewAI
OpenAI Agents SDK
Google ADK
MCP-based architectures
Experience designing and implementing RAG systems.
Knowledge of:
Embeddings
Vector databases
Retrieval algorithms
Context engineering
Experience improving generation quality through prompt engineering and evaluation frameworks.
Strong understanding of Natural Language Processing (NLP) concepts.
Experience building and operating data processing pipelines.
Japanese language proficiency equivalent to JLPT N1.
Preferred Qualifications
Experience designing and operating search platforms using:
Elasticsearch
OpenSearch
Vespa
Similar search technologies
Experience integrating structured knowledge sources or knowledge graphs into AI systems.
Experience designing cloud-native architectures on:
AWS
Google Cloud
Microsoft Azure
Experience processing and normalizing large volumes of unstructured data.
Experience building enterprise-grade AI platforms and services.
Familiarity with large-scale production AI environments.
Ideal Candidate Profile
Passionate about building practical AI products that solve real business challenges.
Strong systems thinker who can balance experimentation with production reliability.
Comfortable making architectural decisions in rapidly evolving technology environments.
Highly collaborative and able to communicate effectively with technical and non-technical stakeholders.
Driven by continuous learning and emerging AI technologies.
Able to thrive in fast-paced, ambiguous environments with a strong sense of ownership.
Focused on delivering measurable business value through AI innovation.
Technology Environment
Programming Languages
Python
TypeScript
Go
AI & Machine Learning
Large Language Models (LLMs)
Retrieval-Augmented Generation (RAG)
AI Agents
Prompt Engineering
NLP
Agent Frameworks
LangChain
CrewAI
OpenAI Agents SDK
Google ADK
MCP Frameworks
Search & Retrieval
Vector Databases
Elasticsearch
OpenSearch
Vespa
Cloud Platforms
AWS
Google Cloud
Microsoft Azure
Compensation & Benefits
Compensation
Annual salary: ¥7.7M – ¥15M
Compensation determined based on experience and expertise
Working Arrangements
Full-flex working hours
No core time
Hybrid work environment
Flexible work style
Professional Development
Self-development allowance for books, certifications, courses, and learning resources
Language learning support
Access to cutting-edge AI and developer productivity tools
Technology Resources
Choice of Mac or Windows device
Enterprise AI tools and productivity platforms
Advanced software development and coding assistant tools
Leave & Holidays
Weekends and national holidays off
Year-end and New Year holidays
Paid annual leave
Sick leave
Birthday leave
Work-life balance leave
Maternity, childcare, and caregiver leave
Benefits
Comprehensive social insurance
Employee stock ownership program
Wellness and vaccination support
Side-business opportunities with approval
Inclusive and diversity-focused workplace culture
Industry
Artificial Intelligence | SaaS | Enterprise Software | Generative AI
Job Function
AI Engineering | Machine Learning Engineering | LLM Engineering | Platform Engineering | Software Development
Career Level
Senior / Staff Engineer / Principal Individual Contributor Track