.. _deploy: Deploy an application ===================== This is a brief and simple tutorial about how you can deploy your app/algorithm on A||go. We try to make it really simple, so, if something looks complicated, send us an email allgo@inria.fr 3 steps are necessary to deploy an app : 1. Create an application on the website 2. Connect by ssh to install your soft 3. Setup your `entrypoint`_ .. note:: Note that the required knowledge you need to have, is only how to install your app. We remind you that A||go only works for application without UI, running on linux, without workflow. You need to add your public SSH key in your profile, so you'll be able to access to your machine later. 1. Create an application ------------------------ In order to create an new app, you need to fill the `webapp creation form `_. You need to specify the application name, the email contact if it's not yourself, a description (in the form of a README) and select if your application should be public or private. .. note:: If you need you can access the "Advanced" tab where you can select a different operating system (the default is the latest stable debian release), the memory limit and default queue type. After creating your app, you'll be redirected to a page where you can deploy a new version of you app. 2. Deploy a new version ----------------------- A||go may host multiple versions of your app. When creating a new job, your users may select a version within the list of available versions. A version is identified with a docker tag (and stored internally as a docker image). You may create a new version either by: - :ref:`running sandbox on A||go ` and installing your software over an SSH connection - building a docker image and :ref:`pushing it to A||go ` .. note:: - Deploying a new version using an existing number overwrites the previous version. - Deleted/overwritten are not expunged immediately, they can be recovered during a grace period (currently 30 days) - Each version has its own **published** flag. If this flag is set, then the version is visible by the other users, otherwise it is only usable by you. .. _deploy-sandbox: 2.1 using a sandbox (over SSH) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The procedure to deploy a new version using a sandbox is the following: 1. create a sandbox 2. open a ssh shell and install your software inside the sandbox 3. provide an entrypoint 4. run sample jobs 5. commit your new version As long as your sandbox is running, you can modifiy its content at will and run new jobs to test it. After commit, the sandbox is stopped, its content is frozen and deployed as a new version for your app. If you want to throw away the changes in the sandbox, you can destroy the sandbox without committing (rollback). .. _sandbox-panel: Create a sandbox """""""""""""""" From your webapp page, follow the link **Create new version** (button |button_new_version|) to proceed to the sandbox panel. From there your can start your sandbox, either from a base image provided by allgo or from an existing version your app (if you want to update a version already deployed). Connect to the sandbox using ssh and install your app """"""""""""""""""""""""""""""""""""""""""""""""""""" Once the sandbox is created, A||go will display the ssh command to be executed to connect to your sandbox. .. note:: You must have a SSH public key configured in `your user profile `_ before connecting to the sandbox. Inside the sandbox you are free to install any package you need (compilers, JRE, python, etc, ...) and of course your software. To keep the image as small as possible, please avoid installing unnecessary packages. The last necessary step before running a job is to provide an entrypoint. .. _entrypoint: The entrypoint """""""""""""" The entrypoint is the executable file used by allgo Allgo when running a job. The user creating the job may provide command line arguments and input files. The arguments are passed to the entrypoint command, and the input files are stored in the current working directory and are listed in the **ALLGO_FILES** environment variable. .. note:: - ALLGO_FILES uses the newline character (``\n``) as record separator between each file name. If you parse this variable in a shell script, then you are recommended to override the `IFS variable `_ (internal field separator) to include only the newline character, **otherwise you app will not be able to process input files containing spaces** (see the example below). - ALLGO_FILES lists the files in the same order as submitted by the user - `Be careful with code injections `_, your users can provide arbitrary inputs (parameters and filenames) you should quote/escape these accordignly. Sample entrypoint: .. code-block:: bash #!/bin/sh IFS=' ' for file in $ALLGO_FILES; do echo "Processing $file" /usr/local/bin/myapp --input "$file" --output "$file.output" "$@" done In this example, myapp is run on each input file listed in ALLGO_FILES and store the result as *FILENAME*.output. The extra argument "$@" is there to forward the command line arguments provided to the entrypoint. Run test jobs """"""""""""" While your sandbox is running you can run test jobs to ensure that your app is working correctly. The job submission form (on your webapp page) includes a special version named *sandbox* for running a job using the current content of the sandbox. It is only visible by you. Commit the new version """""""""""""""""""""" When the app is ready, go to the sandbox panel of your app and submit the commit form. You new version is now deployed. .. _deploy-docker: 2.2 using the registry (push a docker image) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This deployment method is designed to support continuous deployment. The procedure to deploy a new version is the following: 0. generate a deploy token 1. build a docker image containing your application 2. login and push your image to the Allgo registry .. _deploy-panel: Deploy panel """""""""""" The informations and actions needed to push a docker image are hosted on the deploy panel. From your webapp page, follow the link **Create new version** (button |button_new_version|) , then select the **Docker** tab to proceed to reach the deploy panel. Generate a deploy token """"""""""""""""""""""" The deploy token is needed for authenticating with the image registry hosted by Allgo. From the :ref:`deploy-panel` follow the link **Manage deploy tokens**. The next page lists the existing tokens and has a form to create a new token. .. note:: - For better security, Allgo stores the tokens in a hashed format. The plain text is displayed only once and cannot be recovered afterwards (you will need to generate a new token in that case). - The generated token is dedicated to this webapp. If you own multiple webapps, they you will need a separate deploy token for each app. Build a docker image for your app """"""""""""""""""""""""""""""""" There are some guidelines for making an image usable on Allgo. - The image must provide an entrypoint suitable for Allgo (see the :ref:`entrypoint section ` above). Your Dockerfile should contain an ENTRYPOINT directive referencing the command to be executed (if not present, Allgo will use the entrypoint configured in your app properties), example: .. code-block:: Dockerfile ENTRYPOINT ["/usr/local/bin/my_entrypoint"] - The image must be tagged with a name that refers to your app repository hosted by Allgo. The exact repository name to be used is displayed on the :ref:`deploy-panel`. Typically this will look like: ``allgo18.inria.fr/:``. - By default the image's *description* property is blank. If you want to provide a textual description (visible by the users), you may use the **allgo.description** label for that purpose. For example in a context of continuous deployment, you can use this field to track the git commit id used for building the image. Include the following lines in your Dockerfile: .. code-block:: Dockerfile ARG COMMIT_ID="(unknown)" LABEL allgo.description="my webapp (commit $COMMIT_ID)" and build your image with: .. code-block:: bash docker build --build-arg COMMIT_ID="`git show --no-patch --format=%h`" . Login and push your image to the registry """"""""""""""""""""""""""""""""""""""""" You must login with the following credentials: - username: ``token`` - password: The exact command to be used is displayed on the :ref:`deploy-panel`. The deployment is complete when the push succeeds. .. note:: The newly deployed version is immediately published. Your deployment process should run functionnal tests to ensures the image works correctly before pushing it. .. _manage-versions: 3. Manage versions ------------------ From your webapp page, follow the link **List versions** (button |button_version_list|) to list the existing versions. The list is chronological and it includes: - all active versions (those that are currently deployed) - all recently deleted versions (either explicitly deleted or overwritten by another version) From this page, you can: - edit the properties of a version (number, description, published status) - delete a version - restore a recently deleted version .. note: - Deleted versions are available only for a limited period after deletion (currently 30 days). The images are permanently deleted once this period is over. - You may have a name conflict when restoring a deleted version (if there is an active version with the same number). When this happens, you may: - force the restoration and overwrite the active image - or restore the image under a different version number 4. View metrics ------------------ From allgo main web page, follow the **Metric** tab to view all your apllications metrics. You will have the following display options: - Show **one** specific application or **all** - Sort by predifined time period (day, week, month, year) or by specific one - Group results by **day**, **month** or **year** You can export the results in two formats: - JSON - CVS If "**No data**" displayed, please check your time period 5. Import apps from allgo.inria.fr ------------------------------- The old Allgo instance https://allgo.inria.fr/ is scheduled for removal. The applications deployed there can be imported into the new Allgo instance. The import process consists of two steps: 1. importing the app 2. importing the deployed versions 1. Import an application ^^^^^^^^^^^^^^^^^^^^^^^^ Use the `webapp import form `_ (button |button_import| in the apps index). The import is possible if the e-mail address of the app owner is identical and verified, and if there is no existing app with the same name on the new Allgo instance. If you do not meet these condition, please `contact us `_. 2. Import the versions ^^^^^^^^^^^^^^^^^^^^^^ Once the imported app is created, use the version import form (button |button_import| in your app page). The import process is asynchronous. On this page you can start an import and see the status of the previous imports. You can repeat this step multiple time in case you are still updating the original webapp. Thus you can keep both deployments in sync without having to duplicate you work. .. |button_new_version| image:: _static/button_new_version.png :height: 1em .. |button_version_list| image:: _static/button_version_list.png :height: 1em .. |button_import| image:: _static/button_import.png :height: 1em .. Update your application ----------------------- Describe the parameters that can be updated and tags. - Icon - tags - description - default queue