- Location: Melbourne (Hybrid Work Model)
- Salary: $180,000 – $210,000 + Super
- Industry: Financial Services
- Contract Type: Permanent / Full-Time
About the Role
We are seeking a high-caliber Lead AI & Data Platform Engineer to spearhead the evolution of our next-generation data architecture. In this role, you will bridge the gap between advanced Data Engineering and production-grade MLOps, designing resilient, petabyte-scale systems within a highly regulated Financial Services environment.
You will be our internal Databricks Expert , driving the strategy and deployment of AI Agentic workloads, advanced analytics pipelines, and secure cloud infrastructure.
Key Responsibilities 1. Databricks Platform Excellence- Architect and maintain our enterprise Lakehouse infrastructure utilising Delta Lake and Medallion topology (Bronze/Silver/Gold layers).
- Enforce robust data governance and secure discovery across the enterprise using Unity Catalog .
- Build, monitor, and scale production pipelines via Delta Live Tables .
2. Production AI & MLOps Infrastructure
- Deploy, monitor, and scale production-grade ML and AI Agentic workloads.
- Operationalise LLM and Generative AI pipelines utilizing Databricks Mosaic AI , MLflow , Langchain , and Vector Databases .
- Manage corporate feature stores and optimize model serving endpoints.
3. AWS Cloud Architecture & Infrastructure as Code (IaC)
- Design secure, high-performance cloud environments using Terraform.
- Manage intricate AWS configurations including S3 optimization, VPC peering, AWS PrivateLink, and granular IAM roles.
- Integrate cloud-native AI services like AWS Bedrock seamlessly into our data ecosystem.
4. Performance Tuning & Streaming
- Conduct advanced Spark optimization to handle high-concurrency, petabyte-scale data pipelines.
- Tune GPU clusters specifically optimized for heavy AI/ML training and inference workloads.
- Architect real-time streaming pipelines leveraging Kinesis or Kafka.
Your Profile
To be successful in this role, you will bring a blend of elite engineering skills and a deep understanding of modern data-ops.
Technical Requirements:
- Core Languages: Mastery of PySpark , Python , SQL , and Scala .
- AI Frameworks: Strong familiarity with PyTorch , TensorFlow , or Hugging Face .
- DataOps & CI/CD: Proven experience embedding automated testing and CI/CD pipelines into data deployments.
- Industry Experience: Prior experience within Financial Services or a similarly regulated environment is highly desirable (understanding of data privacy, compliance, and risk frameworks).
What’s on Offer
- Competitive Compensation: $180K – $210K base salary, depending on experience.
- Hybrid Flexibility: A healthy balance of remote work and collaborative time in our modern Melbourne CBD office.
- Cutting-Edge Tech Stack: Total backing to utilize the absolute latest in Generative AI, Mosaic AI, and cloud-native tools.
