Evaluate Model

Evaluate Model


Model evaluation in Skyl.ai allows you to evaluate your text multiclass models using a test dataset.  You can automatically compare how well your models are performing by to the true data. This can be done through uploading a completed .csv and viewing the test results by clicking on the dataset.

  1. Select ‘Evaluate Model’ under Machine Learning


  2. Select ‘Start a New Model Evaluation’ on the top right-hand corner of the screen’


  3. Select which deployed model you would like to test and evaluate under “Select a Deployed Model”


  4. Check that your.csv file has the proper schema by selecting “Download Model Evaluation Schema” above the blue box


  5. Upload your .csv file by dragging and dropping into the blue box or choosing it from a local file


  6. Preview your .csv file before evaluation process


  7. Click “Start Evaluation” to start the evaluation process or “Cancel” to cancel the evaluation


  8. Wait until evaluation process will be completed


  9. Review prediction confidence, evaluation result, performance metrics, and class metrics


  10. Download evaluation result as .csv file




Model Inference

Model Inference


Once the training has been completed, a machine learning model would be created which will list under Models on ‘Model Deployment’ page. You can view training summary reports of the model.
In order to start leveraging the power of your newly created model, you need to deploy it first.




To deploy the model go to the ‘Deployment’ tab and select the ‘Deploy’ button




After deploying the Model, Inference API will be available which can be integrated into your application.






Model Training

Model Training


Skyl provides relevant and optimized algorithms to train ML models depending on the ML template. A Text Classification Multiclass model on Skyl is a Natural Language Processing model that classifies or summarizes the idea behind text.

How to train an ML model for Text Classification Multiclass:

  1. Select “Training” under Machine Learning. You will be redirected to the ‘Train’ page


  2. Select ‘Train a new Model’ to start configuring a new model to train


  3. Name the new model and add description.


  4. Select a featureset


  5. Review the featureset details and check your selection conditions.


  6. Select and configure the algorithm.


  7. Select the ‘Train’ button to begin the training process


Initiated training will be displayed on the ‘Training’ page




As the training is in process, user can see the training logs in real-time under the Logs tab.






Creating a Featureset

Creating a Featureset


A featureset is a subset of the dataset with text and their classes which is ultimately used as input for training models.  Using the entire dataset for models can lead to inaccuracy and a higher chance of using unclean data, so we created the featureset to ensure accuracy in more models.

How to create a featureset in text classification multiclass:

  1. Select Machine Learning from the left menu and select “Featureset”. From there, you will be directed to the ‘Featureset’ page


  2. On the top right corner, select “Add Featureset”


  3. Name your new featureset and add its description


  4. From the drop-down menu, select which dataset the featureset will be selected from


  5. On the ‘Feature Selection’ tab, select the categories from the ‘collect’ column. Then, label the column or merge* the collect and label columns

    * Not empty records from label column replace records in collect column.


  6. Define and train the test sets. Skyl provides two options for defining and training test sets. A user can either split the dataset, or explicitly extract from the dataset

    Define the train and test sets with the ‘Split the dataset’ policy:

    1. Choose the selection method from which rows are selected


    2. Enter the number* of records for selection.

      *the number of selected images cannot be bigger than the number of records in dataset


    3. Enter the train ratio (from 0.01 to 1). The train ratio is the proportion of selection rows versus columns that go into the training set. The rest of the images go to the test set


    4. Enter a random seed so that you can reproduce results


    Define train and test sets with the ‘Explicit Extract from the Dataset’ policy:

    1. Choose the selection method and number of records under ‘Train Set’


    2. Enter the number of records and choose selection method test set


    3. Optional- Apply a filter to train and test sets so that you can avoid any unwanted records in the selection process.

    4. Enter a random seed so that you can reproduce results


  7. Select ‘Create Featureset’ to create a new featureset


All created featuresets are displayed on the ‘Featureset’ page.




Each featureset has a status of creation:

  • ‘Completed’ means the featureset is ready to be used for ML training
  • ‘In Progress’ means tells that featureset is being prepared
  • ‘Failed’ means the featureset wasn’t created

Open the Featureset details slider by selecting the proper featureset to see the details and status of each Featureset






Data Labeling using Form Based Collaboration

Data Labeling using Form Based Collaboration


Skyl’s Form Based data labeling is another method to label data which can be done through Skyl Web Collaborator app. You can create a label job to label data and add collaborators. Each collaborator can login to the Web Collaborator App and start labeling data immediately.

The Data Labeling Form Based page displays:

  • Each form based job created for data labeling
  • Trends about the amount of data labeled with Skyl web app

Each job widget displays collaborators and labeling progress.
Users can send messages to all collaborators, or personal messages to other collaborators.
Users can also edit jobs, add collaborators, or remove collaborators.




If user is also a collaborator in a job, he/she can see ‘Start Labeling’ button that will redirect them to Skyl Collaborator App where they can start the labeling job.




The ‘Trends’ widget on the ‘Data Labeling Form Based’ page displays information concerning the amount of data labeled during either the last 7, 15, and 30 days.






Data Labeling using The Skyl Mobile Collaboration Application

Data Labeling using The Skyl Mobile Collaboration Application


The Skyl Mobile Collaboration Application is a tool to do data labeling manually. Any user with access to a project can view and label data.  

The ‘Labeling Mobile’ page provides a dashboard showcasing data about the performance of label jobs of each collaborator, as well as information about the amount of data labeled through the mobile app.

The ‘Labeling Mobile’ page displays every label job created for mobile application.




Each job widget displays collaborators and labeling progress.
Users can send messages to all collaborators, or personal messages to other collaborators.
Users also have the provision to edit a job or add/remove collaborators.

The ‘Trends’ widget in the ‘Labeling Mobile’ page displays information about the amount of data labeled during different time periods, which are -- last 7 days, last 15 days, and last 30 days.




The ‘Download Skyl Collaborator App’ widget displays QR codes for downloading Skyl Mobile App via App Store and Google Play.



Data Labeling Overview

Data Labeling Overview


The ‘Data Labeling Overview’ page provides a dashboard that showcases statistical information your model’s data labeling performance from each data collection source.

The “Trends” widget displays information concerning the amount of data labeled during either the last 7, 15, or 30 days.




The “Statistics” widget displays statistics concerning the labeled data per label source.




The “Label Columns” widget displays the list of all label columns created in the dataset for the purpose of capturing labels. In order to label data, you must create a column that the label of the record will be stored. This column is called the ‘Label Column’. Apart from listing the existing label columns, the “Label Columns” widget allows you to create new label columns.

Steps to create a label column:

  1. Select the “Add Label Column” button which will open a slider


  2. Select the collect column you need to label


  3. Provide a column name and description


  4. Click the “Create” button to complete creating a label column


Please note, users can only create one label column for each collect column.

The “Labeling Outlier Trends” widget displays statistics concerning the outliers for each label column both before and after labeling.  In Skyl, an outlier is a record that has been labelled differently than the value that it was collected for in the ‘collect’ column.






Data Collect using CSV Upload

Data Collect using CSV Upload


Skyl’s CSV Upload is another method of data collection through which a user can upload large amounts of data to the dataset using a .csv file format.

  1. Select “CSV Upload” under the “Data Collection” module


  2. Select the “Upload CSV” link-button


  3. An Upload CSV widget will appear


    Before uploading your .csv file, you must make sure it has the same schema as the dataset

  4. Select “Download Dataset Schema”. From here, you can view the schema of the dataset and compare it with the schema of your existing .csv file. You can also choose to download the dataset schema as a template for data collection


  5. Once you have confirmed that your .csv file has the same schema as your dataset, you can proceed to drag-in your file to the blue section or select the blue section and choose the appropriate file. From here, uploading the dataset begins


  6. To preview the data, upload and choose which data to add to an existing dataset. From here you can also create a new dataset and add it there


  7. Select “Upload” to send your data to existing dataset

  8. Alternatively, you can select “Add as New Dataset”. Proceed by giving your new dataset a name and description, then select the “Upload” button


  9. A widget will pop up that displays the uploaded .csv files




Data Collect using Form Based Collaboration

Data Collect using Form Based Collaboration


Skyl’s Form Based is an another method of data collection which can be done from Skyl Web Collaborator app. You can create a job to collect data with any number of collaborators. Each collaborator can login to the Web Collaborator App to start collecting data.

  1. Select “Form Based” under the “Data Collection” module


  2. Select the “Create a Collaborator Job” link


  3. Configure Collect Job

  4. Monitor job performance from the ‘Form Based’ and ‘Overview’ pages




Data Collect using Skyl Mobile Collaboration

Data Collect using Skyl Mobile Collaboration


Skyl’s Collect Mobile is another method of data collection navigated through the Skyl Mobile app. You can create a job for data collection with any number of collaborators for via mobile application. Each collaborator can install the Skyl Mobile App to start collecting data.

  1. Select “Collect Mobile” under the “Data Collection” module


  2. Select the “Create a Collaborator Job” link


  3. Configure Collect Job

  4. Monitor the job and job performance from ‘Collect Mobile’ and ‘Overview’ pages