About the role
You will build, run and scale Elastic/Observability platforms for a large enterprise SIEM programme. Our
estate covers self-managed and self-hosted Elastic clusters on AWS (EC2 and EKS) and Azure, ingesting
security and observability data at enterprise scale.
• Elastic Engineering designs, automates, onboards data and upgrades the platform.
• Elastic Operations keeps it healthy and stable around the clock, handles tickets and incidents,
and drives RCAs.
Key responsibilities
Scope and depth grow with level. Junior and L1 roles execute; Senior and Lead roles design, own and
lead.
Platform engineering
• Install, configure, size and maintain ELK clusters (self-managed and self-hosted): node roles,
data tiers, ILM, snapshots, CCR/CCS
• Deploy and operate Elastic on EKS (ECK or Helm)
• Plan and execute cluster upgrades, from manual rolling upgrades to fully automated upgrades
with pre-checks and rollback
• Onboard data sources through Fleet, Elastic Agent and integrations, Beats, Logstash and
OpenTelemetry
• Build Logstash pipelines and ingest pipelines for parsing, enrichment and ECS normalisation
• Set up APM and OpenTelemetry agents and collectors
• Configure anomaly detection jobs, Kibana spaces, RBAC and role mappings
• Build dashboards in Kibana and Grafana
Automation and cloud
• Automate platform tasks in Python against Elasticsearch and Kibana APIs
• Build CI/CD pipelines and manage infrastructure with Terraform
• Design AWS log collection and connectivity: VPC Flow Logs, CloudTrail, PrivateLink, Transit
Gateway, VPN, Direct Connect
Operations and support
• Monitor cluster health, ingestion and capacity; perform routine maintenance
• Handle tickets within SLA; triage and resolve issues at L1, escalate or resolve at L2
• Join and lead incident bridges, including Sev1 calls; manage escalations
• Write RCAs and runbooks; track corrective actions to closure
• Senior and Lead roles: mentor engineers, review designs and code, manage shift rosters and
report operational KPIs
Requirements in detail
Candidates should cover most of the areas below. The depth expected rises with the role.
Elastic Stack core
• ELK setup and administration on self-managed and self-hosted clusters
• Cluster troubleshooting: unassigned shards, heap and disk pressure, circuit breakers, slow
queries, node failures and recovery
• Elasticsearch APIs (_cat, cluster, nodes, index and ILM APIs, Query DSL) and Kibana APIs
• KQL, with working knowledge of ES|QL
• Index design, mappings, templates, ILM and data tiers (hot, warm, cold, frozen)
• Cluster upgrades: planning, pre-checks, rolling upgrade, validation and rollback
Data onboarding and parsing
• Fleet, Elastic Agent and its integrations
• Logstash: pipeline configuration, filters, persistent queues and tuning
• Ingest pipelines and data parsing (grok, dissect, processors, ECS mapping)
Observability
• APM setup and configuration
• OpenTelemetry agent and collector setup
• Dashboarding in Kibana and Grafana
• Anomaly detection (machine learning) jobs
Access and security
• Kibana spaces, RBAC, roles and role mapping (SAML, LDAP, AD)
• TLS and certificate management for the stack
Cloud, containers and networking
• AWS basics (EC2, S3, IAM, EKS) and Azure basics
• VPC concepts: subnets, route tables, security groups, NACLs
• Network connectivity options: VPC peering, PrivateLink, Transit Gateway, VPN, Direct Connect,
load balancers
• EKS and Kubernetes operations, including ECK
• Senior level: AWS architecture in depth, including VPC Flow Logs, CloudTrail and cross-account
logging
Automation and DevOps (Senior Engineer and Operations Lead)
• Hands-on Python for automation against Elastic APIs
• CI/CD setup (GitHub Actions, GitLab CI, Jenkins or similar)
• Terraform and infrastructure as code; Git-based workflows
• A track record of automations delivered, e.g. upgrade automation, onboarding automation,
health-check scripts
Operations practice (Operations roles; valued for all)
• Ticket handling within SLA in ServiceNow, Jira or similar
• Incident calls, Sev1 triage and escalation handling
• RCA findings and write-ups; runbook and knowledge-base upkeep
• ITIL incident, problem and change processes
• Willingness to work 24x7 rotational shifts from office
Good to have
• Elastic Certified Engineer or Elastic Certified Observability Engineer
• AWS certification (Solutions Architect Associate or Professional); HashiCorp Terraform
Associate; ITIL v4 Foundation
• Exposure to Elastic Security (SIEM) and detection rules
• Azure log sources and networking (Event Hub, Azure Monitor)
• Linux administration and shell scripting
Qualifications
• B.E./B.Tech/MCA or equivalent in Computer Science or a related field
• Clear written and spoken communication for tickets, RCAs and stakeholder updates