Senior Big Data Engineer
Sydney NSW, Australia
At Data Republic we are delivering technology at the leading edge of the emerging data sector. We work closely with forward-thinking companies, governments and industry influencers around the world to enable a better future for all, powered by data insights. With us, you can be part of a data revolution! We’re looking for people with the curiosity and grit to join a start-up dedicated to changing how the world works with data. Why? We know data has the power to unlock benefits for business and society as a whole, but there are huge consequences when it is misused. That’s why we’re forging a Data Republic where governed access to the world’s data is made possible.
We’re proud to say that we’re backed by some big names (ANZ, Westpac Reinventure, NAB Ventures, Qualgro VC and Qantas Loyalty), but that doesn’t stop us from being a team that cares about startup hustle, big ideas and sharing a beer on a Friday.
We are looking for an experienced Big Data Engineer to help us architect, build and optimise our data platforms to enable us to get ready for the next stage of the Data Republic evolution. You will be lead a small development team from a technical perspective, taking ownership of the requirement, proposing, evaluating, and driving solutions through to completion, delivering software that is ready to production. You will be working across the full software development lifecycle in an Agile environment and take a quality approach to software delivery. We are a fast moving organisation so it’s important you enjoy staying relevant up to date on new technologies that will evolve the data exchange platform to ensure it’s robust, secure and simple to use.
- Design, build and optimise large data extraction, transformation, loading, analysis and secure data matching processes in production
- Design, integrate and document technical components for efficient and cost optimised data extraction and analysis on big data platforms
- Ability to understand and own big data challenges and recommend design patterns commonly used in both Cloud native approach and Hadoop-based deployments
- Work in Java ecosystem and enterprise offering - debugging and profiling tools (jconsole), logging and monitoring tools (log4j, JMX), and security offerings (Kerberos/SPNEGO)
- Mentor and support junior and intermediate engineers
- Working experience with Hadoop, Spark, ETL, MapReduce, Data Lake, Data Warehouse (Required, Minimum 3+ years experience)
- Experience with AWS platform and services
- Knowledge in public cloud Big Data services - AWS EMR, Athena, Glue, Redshift
- Passion for big data technology, design, innovation and latest technologies
- Ability to architect big data solution designs from tactical beginnings to strategic long term solutions
- Experience (and a preference for) working in an Agile environment
- Enjoy working with diverse stakeholders: architects, senior managers, designers, customers,
- investors, data analysts
- Proven delivery of challenging big data technology platform
- and services
- Proven ability to code in Java and/or Python
- Grasp on soft skills such as: facilitation, communication and active listening
- Technical team leadership and mentoring capabilities.
- You love simplicity and code that just works.
Tech you will work with
- Server Technologies: event-pipeline, pub/sub, kafka/kinesis - data streaming, distributed caching, big data computing and storage ( Required, Minimum 2+ years experience )
- Databases: NoSQL - MongoDB, DynamoDb, Cassandra( Nice to have )
- Revision Control: Git (Git Flow)
- Cloud certification
- Team collaboration: Jira, Confluence
- Continuous Integration: Bamboo/GOCD, Stash/Bitbucket
- Containerisation: Docker, Kubernetes
What you will enjoy:
Competitive salary and equity options for all team members
Flexible working arrangements
Parental leave benefits (for both primary and secondary carers)
Sponsored team events and company celebrations
Great central location, currently remote though
Online learning for the whole team
Most importantly, working with clever and curious people that want to evangelise the data space
It's a great time to join Data Republic. We offer:
A well-funded, successful company at an early stage, with the opportunity to grow with the business globally. Excellent remuneration package including an employee stock ownership plan (ESOP) in a fast-growing company. Opportunities for learning and growth on the cutting edge of data and data technologyJoin us in helping the world decide wisely by building a global ecosystem for governed analytics collaborations and data exchanges.
Your application has been successfully submitted.