Incident 2 – Additional details of the attack
This attack targeted LastPass infrastructure, resources, and an employee in a campaign of overlapping activity. The observed tactics, techniques, and procedures (TTPs), as well as the indicators of compromise (IOCs) of the second incident were not consistent with those of the first. While proximal in terms of timeline, it was not initially obvious that the two incidents were directly related.
Our investigation has revealed that the threat actor pivoted from the first incident, which ended on August 12, 2022, but was actively engaged in a new series of reconnaissance, enumeration, and exfiltration activities aligned to the cloud storage environment spanning from August 12, 2022 to October 26, 2022.
The second incident saw the threat actor quickly make use of information exfiltrated during the first incident, prior to the reset completed by our teams, to enumerate and ultimately exfiltrate data from the cloud storage resources.
Alerting and logging was enabled during these events, but did not immediately indicate the anomalous behavior that became clearer in retrospect during the investigation. Specifically, the threat actor was able to leverage valid credentials stolen from a senior DevOps engineer to access a shared cloud-storage environment, which initially made it difficult for investigators to differentiate between threat actor activity and ongoing legitimate activity. Ultimately AWS GuardDuty Alerts informed us of anomalous behavior as the threat actor attempted to use Cloud Identity and Access Management (IAM) roles to perform unauthorized activity.
To access the cloud-based storage resources – notably S3 buckets which are protected with either AWS S3-SSE encryption, AWS S3-KMS encryption, or AWS S3-SSE-C encryption – the threat actor needed to obtain AWS Access Keys and the LastPass-generated decryption keys. The encrypted cloud-based storage services house backups of LastPass customer and encrypted vault data.
- A segregated and secured implementation of an orchestration platform and key-value store used to coordinate backups of LastPass development and production environments with various cloud-based storage resources, or
- A highly restricted set of shared folders in a LastPass password manager vault that are used by DevOps engineers to perform administrative duties in these environments.
Due to the security controls protecting and securing the on-premises data center installations of LastPass production, the threat actor targeted one of the four DevOps engineers who had access to the decryption keys needed to access the cloud storage service.
This was accomplished by targeting the DevOps engineer’s home computer and exploiting a vulnerable third-party media software package, which enabled remote code execution capability and allowed the threat actor to implant keylogger malware. The threat actor was able to capture the employee’s master password as it was entered, after the employee authenticated with MFA, and gain access to the DevOps engineer’s LastPass corporate vault.
The threat actor then exported the native corporate vault entries and content of shared folders, which contained encrypted secure notes with access and decryption keys needed to access the AWS S3 LastPass production backups, other cloud-based storage resources, and some related critical database backups.
As we progress through incident response and as part of our on-going containment, eradication, and recovery activities related to the second incident, we have performed the following actions, with additional work currently being accomplished in scoping and planning:
- With the assistance of Mandiant, we forensically imaged devices to investigate corporate and personal resources and gather evidence detailing potential threat actor activity.
- We assisted the DevOps Engineer with hardening the security of their home network and personal resources.
- We enabled Microsoft’s conditional access PIN-matching multifactor authentication using an upgrade to the Microsoft Authenticator application which became generally available during the incident.
- We rotated critical and high privilege credentials that were known to be available to the threat actor; we continue to rotate the remaining lower priority items that pose no risk to LastPass or our customers.
- We began revoking and re-issuing certificates obtained by the threat actor.
- We analyzed LastPass AWS S3 cloud-based storage resources and applied or started to apply additional S3 hardening measures:
-
- We put in place additional logging and alerting across the Cloud Storage environment with tighter IAM policies enforced.
- We deactivated prior development IAM users.
- We enabled a policy that prevents the creation and use of long-lived development IAM users in the new development environment.
- We rotated existing production service IAM user keys, applied tighter IP restrictions, and reconfigured policies to adhere to least privilege.
- We deleted obsolete service IAM users from the development and production environments.
- We are enabling IAM resource tagging enforcement on accounts for both users and roles with periodic reporting on non-compliant resources.
-
- We rotated critical SAML certificates used for internal and external services.
- We deleted obsolete/unused SAML certificates used for development, services, or third parties.
- We revised our 24x7 threat detection and response coverage, with additional managed and automated services enabled to facilitate appropriate escalation.
- We developed and enabled custom analytics that can detect ongoing abuse of AWS resources.
There are several additional workstreams underway to help secure our customers, which may require them to perform specific actions. Those are detailed in the section titled “What actions should you take to protect yourself or your business.”