Lead Software Developer - Data Infrastructure

Location: Vancouver, British Columbia, Canada

Department: Data Services

Type: Full Time

Min. Experience: Manager/Supervisor

TL;DR: STAT Search Analytics is looking for a talented Lead Software Developer, Data Infrastructure to join our Data services team with a minimum of three years experience who is ready to take on challenges driven by our growth.

We're a Vancouver-based tech company that’s been growing steadily over the last five years, with no signs of slowing down. Our product is a complex, distributed, multi-component system that gathers, analyzes, and delivers high-volume statistical information for our enterprise clients, fresh every day.

You’ll have the opportunity to work with cutting-edge technology while helping to evolve our system and build our team.  STAT is privately owned and 100 percent revenue-funded, with a roster of happy clients that includes some of the world’s largest online companies.
 

OUR TECHNOLOGY
Here’s what you’ll be working with:

Our distributed system runs both synchronously and asynchronously, using some of the newest technologies available. We are based entirely on Amazon Web Services and make use of a wide array of AWS components. We collect approximately 1.5 TB of unstructured data everyday, and process it into approximately 100 GB of structured JSON and 300 million rows of SQL data held across several RDS, DynamoDB and MongoDB instances.

The collection system works on Redis backed queues. The aggregation pipelines are built using Scala and run on AWS EMR clusters.

Our customers are located worldwide and our data acquisition takes place 24 hours a day—which means that we need close to 100 percent uptime across most of our services. Resilience and adaptability are key.

Your main duties and responsibilities:

  • Work with key stakeholders on user stories, acceptance criteria, and other high-level team decisions.
  • Plan, design and coordinate the development of new software components for our data aggregation and storage systems.
  • Work collaboratively in an Agile/Scrum team to plan and execute at a reliable cadence.
  • Coordinate across teams to reuse and architect shared componentry
  • Participate in code reviews and other technical discussions
  • Mentor junior developers and onboard new team hires
  • Work with the team to manage the operation of STAT's data processing and storage systems ensuring that they are running smoothly at all time, upgrading and developing maintenance procedures as needed.
  • Draft technical documentation of our software, development environments, production environments and procedures.
  • Lead team level meetings and discussion for estimation, software design, software development, and code review.
  • Discuss progress on work being undertaken by yourself and the Data Services team on a regular (daily and weekly) basis with the rest of the development team and other key stakeholders.
  • Discuss and participate in the ongoing refinement of our software development process.

 

REQUIREMENTS

You’ve got experience

You should have at least three years of experience working with data pipelines in MapReduce, Streaming, Batching or similar frameworks in a large scale production environment, and should be comfortable leading and mentoring developers. This role is very hands-on and will be focused on system architecture, software design, research, testing and development.  

You’ve got education

At a minimum, you should have a post-secondary diploma or equivalent degree in computer science, engineering or a related discipline.

You’ve got leadership chops

This is a lead role, and as such, you’ll need the necessary skills to maintain a high performing team of developers.

  1. Proven ability to take complex information from multiple sources, distilling and expressing the essential details in clear writing and diagrams.
  2. Strong communicator who likes to teach and broaden the skills of others
  3. Must be pragmatic and well-organized, creating a team structure that is resilient and scalable

You’ve got skills

We are looking for people who are confident working in our technical landscape, including distributed technologies and software-as-a-service. In order to be eligible for this role, we need you to display competency in the following areas:

  1. Building data pipelines using AWS EMR, AWS Data Pipeline, ECS, SQS or similar technologies
  2. BDD or TDD development in an Agile-like process
  3. Scala or other common data pipeline languages like Java, Haskell, etc.
  4. Knowledge of Hadoop ecosystem and similar ETL based frameworks
  5. Ruby or other scripting languages like Python, Go etc.
  6. SQL databases like MySQL, Postgres, MS SQL Server, etc.
  7. NoSQL databases like MongoDB, DynamoDB, Redshift etc.
  8. Caches like Redis, Memcache, etc.
  9. Queue Management Systems like AWS SQS, Resque-Redis, ZeroMQ, Celery, etc.
  10. Source control like Git

PERKS
A few side benefits of working here:

  • Three week paid vacation, plus everyone gets an extra two-week paid vacation leading up to New Year’s.
  • Shorter summer hours to take advantage of that elusive Vancouver sunshine.
  • Free gym memberships and transit pass subsidy for public transit users.
  • Benefits package including dental, optometry, prescriptions, massage, and physio.
  • Plenty of opportunities for further education and career growth.

CULTURE

Our Development culture at STAT is an open and collaborative agile like workflow. We believe that the best way to solve challenging engineering problems is to bring the most inventive ideas together through discussion and then implement solutions through an iterative process that involves constant communication and feedback.

We believe the best way to foster growth and creativity is to bring multiple minds together on an issue and then make one individual responsible for ultimately solving it. We are never hesitant to ask for help/advice and never too busy to offer the same.

We're a small, highly focused team of experts, doing world-class, global-scale data mining and classification. We're also building interfaces around this data that allow our enterprise customers to leverage the insights we provide in order to grow their businesses. Our customers love us.

Our founder & CEO is a software developer at heart, so we have a development-focused culture. We're a 100% revenue-funded private company that is focused on delivering excellent work, and having a good time while doing it.

Some reasons why you’d want to work with us:

  • You want to play a big part in growing a young, profitable company while working alongside people who are smart, easy-going, and helpful.
  • You do your best in a setting where excellent work is what’s valued—not long hours. (A good work-life balance is just how we do things.)
  • You’d like to work somewhere where you can wear your old hoodie—or your crisp new suit—and nobody is going to mind.
  • You’re looking for a big challenge that involves lots of variety, collaboration, inventiveness, and on your toes thinking.
  • You can see yourself joining a tight team in an open and collaborative Railtown office where you might sometimes see a dog zipping around.
Apply for this Position
* Required fields
First name*
Last name*
Email address*
Location
Phone number*
Resume*

Attach resume as .pdf, .doc, .docx, .odt, or .rtf (limit 5MB) or paste resume

Paste your resume here or attach resume file

Cover Letter*
In 150 characters or fewer, tell us what makes you unique. Try to be creative and say something that will catch our eye!*