Evaluate Model

Evaluate Model


Model evaluation in Skyl.ai allows you to evaluate your text multilabel models using a test dataset.  You can automatically compare how well your models are performing by to the true data. This can be done through uploading a completed .csv and viewing the test results by clicking on the dataset.

  1. Select ‘Evaluate Model’ under Machine Learning


  2. Select ‘Start a New Model Evaluation’ on the top right-hand corner of the screen’


  3. Select which deployed model you would like to test and evaluate under “Select a Deployed Model”


  4. Check that your.csv file has the proper schema by selecting “Download Model Evaluation Schema” above the blue box


  5. Upload your .csv file by dragging and dropping into the blue box or choosing it from a local file


  6. Preview your .csv file before evaluation process


  7. Click “Start Evaluation” to start the evaluation process or “Cancel” to cancel the evaluation


  8. Wait until evaluation process will be completed


  9. Review prediction confidence, evaluation result, performance metrics, and class metrics


  10. Download evaluation result as .csv file




Evaluate Model

Evaluate Model


Model evaluation in Skyl.ai allows you to evaluate your text multiclass models using a test dataset.  You can automatically compare how well your models are performing by to the true data. This can be done through uploading a completed .csv and viewing the test results by clicking on the dataset.

  1. Select ‘Evaluate Model’ under Machine Learning


  2. Select ‘Start a New Model Evaluation’ on the top right-hand corner of the screen’


  3. Select which deployed model you would like to test and evaluate under “Select a Deployed Model”


  4. Check that your.csv file has the proper schema by selecting “Download Model Evaluation Schema” above the blue box


  5. Upload your .csv file by dragging and dropping into the blue box or choosing it from a local file


  6. Preview your .csv file before evaluation process


  7. Click “Start Evaluation” to start the evaluation process or “Cancel” to cancel the evaluation


  8. Wait until evaluation process will be completed


  9. Review prediction confidence, evaluation result, performance metrics, and class metrics


  10. Download evaluation result as .csv file




Model Inference

Model Inference


Once the job training has been completed, a machine learning model would be created which will list under Models on ‘Model Deployment’ page. You can view training summary reports of the model.




In order to start leveraging the power of your newly created model, you need to deploy it first.




To deploy the model go to the ‘Deployment’ tab and select the ‘Deploy’ button




After deploying the Model, Inference API will be available which can be integrated into your application.






Model Training

Model Training


Skyl provides relevant and optimized algorithms to train ML models depending on the ML template. A Text Extraction Named-Entity Recognition model on Skyl is a Natural Language Processing model that identifies entities of interest within the text.

How to train an ML model for Text Extraction Named-Entity Recognition:

  1. Select “Training” under Machine Learning. You will be redirected to the ‘Train’ page


  2. Select ‘Train a new Model’ to start configuring a new model to train


  3. Name the new model and add description.


  4. Select a featureset


  5. Review the featureset details and check your selection conditions.


  6. Select and configure the algorithm.


  7. Select the ‘Train’ button to begin the training process


Initiated training will be displayed on the ‘Training’ page




As the training is in process, user can see the training logs in real-time under the Logs tab.






Creating a Featureset

Creating a Featureset


A featureset is a subset of the dataset with text and their classes which is ultimately used as input for training models.  Using the entire dataset for models can lead to inaccuracy and a higher chance of using unclean data, so we created the featureset to ensure accuracy in more models.

How to create a featureset in text extraction (named-entity recognition):

  1. Select Machine Learning from the left menu and select “Featureset”. From there, you will be directed to the ‘Featureset’ page


  2. On the top right corner, select “Add Featureset”


  3. Name your new featureset and add its description


  4. From the drop-down menu, select which dataset the featureset will be selected from


  5. On the ‘Feature Selection’ tab, select the categories from the ‘collect’ column. Then, label the column or merge* the collect and label columns

    * Not empty records from label column replace records in collect column.


  6. Define and train the test sets. Skyl provides two options for defining and training test sets. A user can either split the dataset, or explicitly extract from the dataset

    Define the train and test sets with the ‘Split the dataset’ policy:

    1. Choose the selection method from which rows are selected


    2. Enter the number* of records for selection.

      *the number of selected records cannot be bigger than the number of records in dataset


    3. Enter the train ratio (from 0.01 to 1). The train ratio is the proportion of selection rows versus columns that go into the training set. The remaining are transferred to the test set


    4. Enter a random seed so that you can reproduce results


    Define train and test sets with the ‘Explicit Extract from the Dataset’ policy:

    1. Choose the selection method and number of records under ‘Train Set’


    2. Enter the number of records and choose selection method test set


    3. Optional- Apply a filter to train and test sets so that you can avoid any unwanted records in the selection process.

    4. Enter a random seed so that you can reproduce results


  7. Select ‘Create Featureset’ to create a new featureset


All created featuresets are displayed on the ‘Featureset’ page.




Each featureset has a status of creation:

  • ‘Completed’ means the featureset is ready to be used for ML training
  • ‘In Progress’ means tells that featureset is being prepared
  • ‘Failed’ means the featureset wasn’t created

Open the Featureset details slider by selecting the proper featureset to see the details and status of each Featureset






Data Labeling using Form Based Collaboration

Data Labeling using Form Based Collaboration


Skyl’s Form Based data labeling is another method to label data which can be done through Skyl Web Collaborator app. You can create a label job to label data and add collaborators. Each collaborator can login to the Web Collaborator App and start labeling data immediately.

The Data Labeling Form Based page displays:

  • Each form based job created for data labeling
  • Trends about the amount of data labeled with Skyl web app

Each job widget displays collaborators and labeling progress.
Users can send messages to all collaborators, or personal messages to other collaborators.
Users can also edit jobs, add collaborators, or remove collaborators.




If user is also a collaborator in a job, he/she can see ‘Start Labeling’ button that will redirect them to Skyl Collaborator App where they can start the labeling job.




The ‘Trends’ widget on the ‘Data Labeling Form Based’ page displays information concerning the amount of data labeled during either the last 7, 15, and 30 days.






Data Labeling using The Skyl Mobile Collaboration Application

Data Labeling using The Skyl Mobile Collaboration Application


The Skyl Mobile Collaboration Application is a tool to do data labeling manually. Any user with access to a project can view and label data.  

The ‘Labeling Mobile’ page provides a dashboard showcasing data about the performance of label jobs of each collaborator, as well as information about the amount of data labeled through the mobile app.

The ‘Labeling Mobile’ page displays every label job created for mobile application.




Each job widget displays collaborators and labeling progress.
Users can send messages to all collaborators, or personal messages to other collaborators.
Users also have the provision to edit a job or add/remove collaborators.

The ‘Trends’ widget in the ‘Labeling Mobile’ page displays information concerning the amount of data labeled during different time periods, either the last 7, 15, or 30 days.




The ‘Download Skyl Collaborator App’ widget displays QR codes for downloading Skyl Mobile App via App Store and Google Play.



Data Labeling Overview

Data Labeling Overview


The ‘Data Labeling Overview’ page provides a dashboard that showcases statistical information about your model’s data labeling performance from each data collection source.

The “Trends” widget displays information concerning the amount of data labeled during either the last 7, 15, or 30 days.




The “Statistics” widget displays statistics concerning the labeled data per label source.




The “Label Columns” widget displays the list of all label columns created in the dataset for the purpose of capturing labels. In order to label data, you must create a column that the label of the record will be stored. This column is called the ‘Label Column’. Apart from listing the existing label columns, the “Label Columns” widget allows you to create new label columns.

Steps to create a label column:

  1. Select the “Add Label Column” button which will open a slider


  2. Select the collect column you need to label


  3. Provide a column name and description


  4. Click the “Create” button to complete creating a label column


Please note, users can only create one label column for each collect column.

The “Labeling Outlier Trends” widget displays statistics concerning the outliers for each label column both before and after labeling.  In Skyl, an outlier is a record that has been labelled differently than the value that it was collected for in the ‘collect’ column.






Configuring Dataset

Configuring Dataset


Configuring a dataset prepares your project’s data before entering it, ensuring clean and accurate data.  In Skyl, you can enter the name of your dataset and a short description for your dataset. You can also create labels for your dataset and assign values for each label.

  1. After choosing a project name and description, you will be directed to the ‘Design the Dataset for your Project’ page. Here, name and describe your dataset.

    Sample dataset: ‘Bio Entity’


  2. Enter the desired named-entity values, which will be used to label pieces of text. In this case, the text could be labeled as either “cell_line”, “cell_type”, “dna”, or “protein”


  3. Select ‘Submit’ after entering the details

  4. Select ‘Cancel’ to design your dataset later. In this case, the details of the project will not be saved and you will be directed to the Skyl Welcome Page

  5. You can select the card for this project at any point to continue designing your dataset



Data Collect using CSV Upload

Data Collect using CSV Upload


Skyl’s CSV Upload is another method of data collection through which a user can upload large amounts of data to the dataset using a .csv file format.

  1. Select “CSV Upload” under the “Data Collection” module


  2. Select the “Upload CSV” link-button


  3. An Upload CSV widget will appear


    Before uploading your .csv file, you must make sure it has the same schema as the dataset

  4. Select “Download Dataset Schema”. From here, you can view the schema of the dataset and compare it with the schema of your existing .csv file. You can also choose to download the dataset schema as a template for data collection


  5. Once you have confirmed that your .csv file has the same schema as your dataset, you can proceed to drag-in your file to the blue section or select the blue section and choose the appropriate file. From here, uploading the dataset begins


  6. Once you drop the file for upload, the preview of the file will be displayed if there were no errors in the file schema i.e. the schema of the csv file you uploaded matched with the dataset. Here, you can upload the data in the file to current dataset or create a new dataset.


  7. Select “Upload” to send your data to existing dataset

  8. Alternatively, you can select “Add as New Dataset”. Proceed by giving your new dataset a name and description, then select the “Upload” button


  9. A widget will pop up that displays the uploaded .csv files