At Lumenalta, we create impactful software solutions that drive innovation and transform businesses. Since 2000, we’ve partnered with visionary leaders to build cutting-edge tech, solve complex challenges, and deliver results faster through our elite teams and tech-driven approach. Join us in shaping the future of technology.
What you’ll do
- Architect end-to-end pipelines using PySpark, SQL, DLT, and dbt for batch and streaming use cases.
- Design governed lakehouse solutions using Unity Catalog, Delta Lake, and scalable architecture best practices.
- Leverage cutting-edge Databricks features including:
- Unity Catalog for access control and lineage
- Delta Live Tables for reliable streaming
- Liquid Clustering for performance tuning
- System Tables for monitoring and cost insights
- Serverless Compute for elastic scalability
- AI/BI Genie for natural language insights
- Query Federation and Delta Sharing for unified access
- Automate everything: Use Terraform, Databricks CLI, and REST APIs to provision and manage infrastructure.
- Streamline CI/CD: Build pipelines for notebooks, workflows, DLT, and ML models using GitHub Actions, Jenkins, or Azure DevOps.
- Collaborate cross-functionally: Translate complex business needs into simple, scalable data architecture.
- Champion governance and security: Enforce fine-grained access, logging, and compliance using Unity Catalog and platform controls.
- Stay ahead of the curve: Explore and guide adoption of new Databricks features like Lakehouse Federation, GenAI integrations, and Model Serving.
- Mentor and uplift: Share knowledge, contribute to internal standards, and help raise the bar for data engineering across the team.
What you bring
- 6+ years of experience in data engineering/architecture, including 3+ years hands-on with Databricks in production.
- Deep understanding of Databricks features and architecture, especially: Unity Catalog, Delta Live Tables, Liquid Clustering, Serverless Compute, Photon Engine, System Tables, Query Federation and Delta Sharing
- Strong cloud knowledge—AWS is your home base (S3, IAM, Glue, Redshift, Lambda, Athena, ECS/EKS).
- Infrastructure automation experience with Terraform, CLI tools, and REST APIs.
- CI/CD pro with GitHub Actions, Jenkins, Azure DevOps, or similar.
- Fluency in PySpark, SQL, and Python; experience with dbt, MLflow, Spark internals a plus.
- Practical knowledge of data governance, lineage, access controls, and compliance in cloud-native environments.
- Clear communicator with strong architectural instincts and attention to detail.
Nice to have
- Databricks certifications (Data Engineer Pro, Machine Learning Pro, Lakehouse Architect)
- Experience with Model Serving, Feature Store, Vector Search, or GenAI use cases
- Comfort integrating with BI tools like Tableau, Power BI, and Databricks SQL
- Past work on migrations from Hadoop, Teradata, Informatica, or similar
- Exposure to Databricks on Azure or GCP
- Familiarity with data observability tools
Why join Lumenalta?
- You’ll work on high-impact projects that power real-time analytics, AI, and innovation at scale.
- You’ll be surrounded by smart, supportive people who care about doing great work and helping each other grow.
- You’ll stay on the forefront of Databricks capabilities and cloud-native design patterns.
- We offer competitive compensation, flexibility, and the chance to shape meaningful solutions for forward-thinking clients.
What's it like to work at Lumenalta?
Ongoing recruitment – no set deadline.
Salary
Salary range: $75k - $140k annually, with final compensation determined by your qualifications, expertise, experience, and the role's scope.
Benefits
In addition to competitive pay, we offer a variety of benefits to support your professional and personal growth, including:
- Flexible working hours in a remote environment.
- Health insurance (medical and dental) for W2 Employees.
- 401K Contribution.
- A professional development fund to enhance your skills and knowledge.
- 15 days of paid time off annually.
- Access to soft-skill development courses to further your career.
Position Details
This is a full-time position requiring a minimum of 40 hours per week, Monday through Friday.
At Lumenalta, we are committed to creating an environment that prioritizes growth, work-life balance, and the diverse needs of our team members.