Save the script, upload it to S3 and take note of its URL, such as s3://scripts/taskrunner.sh The second line then executes the JAR file with standard java -jar. if the ec2-user won’t be able to save the file to just any folder on the EC2 instance so it’s wise to select the default home directory of that user. not admin which can lead to authorisation problems. Data Pipeline will access the new EC2 instance under the default “ec2-user” username, i.e. The first line calls upon the Amazon CLI to copy a file located on S3 into the /home/ec2-user/ folder on the generated EC2 machine. Java -jar /home/ec2-user/SimpleModelJarForDataPipeline.jar You may see some strange behaviour such as the JAR file is downloaded but then it cannot be located.Ĭreate a bash script with the following 2 rows:Īws s3 cp s3://bucket-for-blog/SimpleModelJarForDataPipeline.jar /home/ec2-user/SimpleModelJarForDataPipeline.jar The linefeed character won’t be properly recognised by the Linux EC2 instance trying to run the script. Do not create the script in a Windows-based text editor like Notepad or Notepad++. The accompanying bash script is extremely simple but make sure you create it in a Linux-based editor or, if you want to edit the script in Windows, in a Windows-compatible bash script editor. So say you have the following compiled Java application in S3: You’ll need at least 2 elements: the JAR file to be executed and a bash script which loads the JAR file onto the EC2 instance created by Data Pipeline and then executes it. It allows you to run a Linux bash script on an EC2 instance – or an Elastic MapReduce instance, but I didn’t see any use of that in my case. The activity type to pick in this case is ShellCommandActivity.
I won’t go into any of these – I’m not sure how to use them and I’d like to concentrate on the solution to the problem outlined above. It has several so-called activity types, like CopyActivity, HiveActivity or RedShiftCopyActivity. In that case Data Pipeline is an option to consider. You’d like to execute this file automatically with logging and retries.
The JAR file can carry out ANY activity – it can range from printing “Hello world” to the console window to a complex application that interacts with databases and/or other Amazon components to perform some composite action.You have a compiled JAR file saved on S3.You are working on a project within the Amazon web services environment.This may not be something you do every single day but I really could have used an example when I went through this process in a recent project. In this post we’ll go through a very specific example of using Data Pipeline: run an arbitrary JAR file from an EC2 instance through a bash script. Amazon Data Pipeline helps you automate recurring tasks and data import/export in the AWS environment.
0 kommentar(er)
