Bioinformatics Pipelines in AWS

Deploying Nextflow experiments in AWS

Mark Preston

CEO at Prismea

"Thanks to Crunchloop’s efforts, the app met testing requirements and received positive feedback from end clients and prospects. The team seamlessly integrated their process to the client's methodology, and internal stakeholders were impressed with the vendor's work ethic."

Read full review on Clutch
service icon

Service

Project Development
team icon

Team

2 Developers
technology icon

Technology

AWS
industry icon

Industry

Health

Who is

Prismea

?

Prismea specializes in providing bioinformatics, cloud infrastructure, and compliance services for biotech companies. They excel in data analytics, cloud integration, and automation of scientific data workflows. Their solutions ensure security, privacy, and intellectual property compliance, enabling clients to enhance team skills, scale infrastructure, and maintain robust compliance processes.

Challenge

Prismea approached us with the need for an automated solution to deploy scalable bioinformatics pipelines in AWS. They required the implementation to be done using CloudFormation and needed the infrastructure to be ephemeral, ensuring environments could be deployed for experiments and then destroyed to minimize costs. Key requirements included:

• Ephemeral infrastructure for cost-efficiency

• Security measures (limiting internet access, independent networks)

• Minimum permissions granted within AWS

• Use of Nextflow for bioinformatics algorithms

Solution

Streamlined Genomics with AWS and Nextflow

To address Prismea’s needs, we designed a robust solution based on the architecture outlined in AWS Genomics Workflows. We utilized S3 for data storage, where data readiness was indicated by an S3 event triggering further processing. This event activated a Lambda function, which subsequently called AWS Batch to initiate a Nextflow pipeline. The Nextflow pipeline handled the bioinformatics algorithms, processing the data and writing the results back to S3. This entire workflow ensured that Prismea’s data was processed efficiently and securely.

Simplifying AWS with a Custom CLI

Furthermore, we developed a custom CLI to abstract the complexities of AWS operations from the user, allowing Prismea’s team to manage and deploy their bioinformatics experiments seamlessly. This CLI facilitated easy interaction with the AWS environment, ensuring that even users without deep AWS knowledge could efficiently utilize the infrastructure.

Strengthening Security with Least Privilege and Isolation

The architecture emphasized security by limiting internet access and creating independent networks for different components. We also enforced the principle of least privilege, granting minimum necessary permissions for all AWS interactions, thereby enhancing the security posture of the solution.

Things we love

The implementation provided Prismea with a scalable, secure, and cost-effective solution for running bioinformatics pipelines. The solution’s ephemeral infrastructure allowed for efficient resource usage, and the custom CLI facilitated easy deployment and management of experiments. The architecture ensured data security and adhered to AWS best practices, making it an optimal solution for Prismea’s needs.

Get in Touch

Do you have a project in mind? If you have any questions about our services, or just want to say hello, we're here to help. Fill out the form, and we'll get back to you as soon as possible!

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.