Full-Time Position - Toronto
Hiring a full-time Senior Hadoop Infrastructure Engineer in Toronto, ON! This role will be an essential member of our client's DaaS team. You will have a mandate of capturing, storing and processing data reliably at scale. This Team makes this data available for a large set of products that are used for internal and external services. The core infrastructure that powers this platform operates at a scale of speed, performance, and complexity that few others can claim.
The issues our client faces with large-scale data storage, low-latency retrievals, high volume requests and high availability are common yet complex.
Perks You'll Love
- Join a cutting edge, go big or go home team with transparent culture
- Annual performance and team party
- Compensation plan includes Apple Macbook Pro/Air & Smartphone
What You'll Be Doing
- You will participate in and build tools to diagnose and fix complex distributed systems handling petabytes of data & drive opportunities to automate infrastructure, deployments, and observability of data services.
- You will test, monitor, administer, optimize and operate multiple Hadoop / Spark clusters across cloud providers - AWS, GCP, Aliyun and on premise data centers, primarily in Python, Java and Scala.
- Investigate emerging technologies in Hadoop ecosystem that relate to our needs and implement those technologies.
- Partner with Hadoop developers in building best practices for Data Warehouse and analytics environment. Share an on-call rotation and handle service incidents.
- Working with Big Data tools and building high performance, high throughput, and distributed data pipeline and big data platform with Hadoop, Spark, Kakfa, Hive, and Presto.
What You're Bringing
- Minimum 3+ years of handling services in a large scale distributed systems environment, preferably Hadoop.
- Hands on experience with Hadoop (or similar) ecosystem - Yarn, Hive, HDFS, Spark, Presto, Parquet, HBase
- Experience with workflow management (Airflow, Oozie, Azkaban)
- Familiarity with systems management tools (Puppet, Chef, Capistrano, etc)
- Knowledge of Linux operating system internals, file systems, disk/storage technologies and storage protocols and networking stack.
- Proven knowledge of systems programming (bash and shell tools) and/or at least one scripting language (Python, Ruby, Perl).
- Be adaptable and able to focus on the simplest, most efficient & reliable solutions.
We Are An Equal Opportunity Employer
All qualified applicants will receive equal consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.
Canadian Work Authorization
All applicants MUST be authorized to work in Canada. Any applicants not meeting this criteria will not be notified and will not be considered eligible for the position.
Apply for this Job