API

Module: services/basicWorkflowService

Description

This is a job services that executes local NCBI blast by either excuting NCBI.blast or Sim.blast, defined by the job.

This job service is functionally equivelant to galaxyService, which does blast search through Galaxy API.

Job submission example:

var postData = {
      service: "jblast",
      dataset: "sample_data/json/volvox",
      region: ">ctgA ctgA:44705..47713 (- strand) class=remark length=3009\nacatccaatggcgaacataa...gcgagttt",
      workflow: "NCBI.blast.workflow.js"
  };
$.post( "/job/submit", postData , function( result ) {
    console.log( result );
}, "json");

Configuration:

 jblast: {
    // The subdir where blast results will be deposited (i.e. ``sample_data/json/volvox/jblastdata``)
    blastResultPath: "jblastdata",

    // The category for successful blast results in the track selector
    blastResultCategory: "JBlast Results",

    // Track template of the blast result track that will be inserted in trackList.json
    trackTemplate: "jblastTrackTemplate.json",

    // Type of file that will be imported for processing blast.
    import: ["blastxml"],


    // BLAST profiles
    // blast profiles are parameter lists that translate to blastn cli parameters sets
    // (i.e. for "remote_htgs" would translate to "blastn -db htgs -remote")
    // These will override any default parameters defined in ``blastjs``
    //
    // Blast profiles generally apply to basicWorkflowService only
    // and do no apply to galaxyService.
    //
    // Our example uses a subset of htgs, an NCBI curated blast database.
    // So, it is our default profile.
    defaultBlastProfile: 'htgs',
    blastProfiles: {
        'htgs': {
            'db': 'htgs'
        },
        'remote_htgs': {
            'db': 'htgs',
            'remote': ""
        }
    }
},
// list of services that will get registered.
services: {
    'basicWorkflowService':     {name: 'basicWorkflowService',  type: 'workflow', alias: "jblast"},
    'filterService':            {name: 'filterService',         type: 'service'},
    'entrezService':            {name: 'entrezService',         type: 'service'}
},

Job queue entry example:

{
  "id": 145,
  "type": "workflow",
  "progress": "0",
  "priority": 0,
  "data": {
    "service": "jblast",
    "dataset": "sample_data/json/volvox",
    "region": ">ctgA ctgA:44705..47713 (- strand) class=remark length=3009\nacatccaatggcgaacataagcgagttttgt...tggccc",
    "workflow": "NCBI.blast.workflow.js",
    "name": "NCBI.blast.workflow.js",
    "sequence": {
      "seq": "ctgA",
      "start": "44705",
      "end": "47713",
      "strand": "-",
      "class": "remark",
      "length": "3009"
    },
    "blastData": {
      "name": "JBlast",
      "blastSeq": "/var/www/html/jbconnect/node_modules/jbrowse//sample_data/json/volvox/jblastdata/blast_region1517044304838.fa",
      "offset": "44705"
    },
    "seqFile": "http://localhost:1337/jbrowse/sample_data/json/volvox/jblastdata/blast_region1517044304838.fa",
    "blastOptions": {
      "db": "htgs"
    },
    "blastOptionFile": "/tmp/blast_option1517044304843.json"
  },
  "state": "failed",
  "promote_at": "1517044302842",
  "created_at": "1517044302842",
  "updated_at": "1517044310134",
  "createdAt": "2018-02-01T05:38:27.406Z",
  "updatedAt": "2018-02-01T05:38:27.406Z"
},

Function: validateParams

job service validate

validateParams(params)
Arguments:
  • params (object) – parameters
Return int:

0 if successful

Function: generateName

job service generate name.

generateName(params)
Arguments:
  • params (object) – parameters
Return string:

string job name

Function: get_workflows

Enumerate available workflow scripts

get_workflows(req, res)
Arguments:
  • req (object) – request
  • res (object) – response

Function: get_hit_details

get_hit_details()

Function: beginProcessing

Job service - job execution entry point

beginProcessing(kJob)
Arguments:
  • kJob (object) – reference to the kue job object

Function: determineBlastProfile

determineBlastProfile()

Function: beginProcessing2

beginProcessing2()

Module: services/blastxml2json

Local Navigation

Description

Convert BlastXML to JSON (not a straight conversion) This script not only converts the XML to json, it flattens the hits per hsp where there are multiple hsp.

Creates an indexed list by feature ID. Essentially, it simpifies the hit array into an associative array and makes it indexed by key, where key is <Hit_id>;<Hsp_num>

Function: convert

Perform the conversion operation

convert()
Arguments:
  • convert() – kJob - kue job object
  • convert() – trackJson
  • convert() – cb - callback function

Member: err:

Module: services/entrezService

Description

This job service enables accession value lookup utilizeing Entrez API.

Ref: Entrez

Function: init

Initialize the module

init(req, res, cb)
Arguments:
  • req (object) – request
  • res (object) – response
  • cb (function) – callback function

Function: lookup_accession

This does an esummary lookup (using Entrez api), adding the link field into the result.

lookup_accession(req, res)
Arguments:
  • req (object) – request
  • res (object) – response

Module: services/filter

Description

Supporting methods for the filterService jservice.

Function: filterInit

create initial filter settings file

filterInit(kJob)
Arguments:
  • kJob (object) – kue job object
  • filterInit(kJob) – cb - callback

Function: getFilterSettings

get filterData

getFilterSettings(requestData, cb)
Arguments:
  • requestData (object) – eg. { asset: ‘jblast_sample’, dataset: ‘sample_data/json/volvox’ }
  • cb (object) – function(filterData)
::
eg. filterData: {
contig: “ctgA”, score: {type: ‘abs’, min: 58, max: 593, val: 421 }, evalue: { type: ‘exp’, min: 5.96151e-165, max: 0.000291283, val: 0.000291283 }, identity: { type: ‘pct’, min: 78, max: 100, val: 78 }, gaps: { type: ‘pct’, min: 0, max: 13, val: 13 }

}

Function: writeFilterSettings

write new data to filter settings file, given requestData

writeFilterSettings(requestData, cb)
Arguments:
  • requestData (object) – eg. { asset: ‘jblast_sample’, dataset: ‘sample_data/json/volvox’, filterParams: filterData }
  • cb (object) – updated filterData function(filterData)
::
eg. filterData: {
contig: “ctgA”, score: {type: ‘abs’, min: 58, max: 593, val: 421 }, evalue: { type: ‘exp’, min: 5.96151e-165, max: 0.000291283, val: 0.000291283 }, identity: { type: ‘pct’, min: 78, max: 100, val: 78 }, gaps: { type: ‘pct’, min: 0, max: 13, val: 13 }

}

Function: applyFilter

Based on the filterData, generate a new gff3 file. Also announces the track to subscribed clients.

applyFilter(filterData, requestData)
Arguments:
  • filterData (object) – the output of writeFilterSettings or getFilterSettings.
  • requestData (object) – eg. { asset: ‘jblast_sample’, dataset: ‘sample_data/json/volvox’ }

callback:

cb({
   totalFeatures: x,               // total number of features
   filteredFeatures: x             // filtered features.
})

Function: getHitDataFiltered

getHitDataFiltered()

Function: _announceTrack

_announceTrack()

Function: getHitDetails

return hit details given hit key, including all HSPs of the original hit. The hit key looks like this “gi-402239547-gb-JN790190-1–3” Separate the hit id ==> “gi-402239547-gb-JN790190-1–” (basically remove the last number) Returns multiple HSPs for each hit id: data for “gi-402239547-gb-JN790190-1–1”, “gi-402239547-gb-JN790190-1–2”…

getHitDetails(hitkey, cb)
Arguments:
  • hitkey (string) – eg. “gi-402239547-gb-JN790190-1–3”
  • cb) (getHitDetails(hitkey,) – dataSet - eg. “sample_data/json/volvox”
  • cb (function) – callback

Function: getHighestPct

getHighestPct()

Function: getHitId

getHitId()

Constant: _:

Module: services/filterService

Description

This jservice provides restful APIs for processing filter requests.

Function: set_filter

Based on new filter settings provided by the caller, updates the associated filtersettings file and the resulting GFF3 file containing filtered features.

REST Request:
POST /service/exec/set_filter
set_filter(req, res)
Arguments:
  • req (object) – request
 
req.body = {
  filterParams: {score:{val: 50}, evalue:{val:-2}...
  dataSet: (i.e. "sample_data/json/volvox" generally from config.dataRoot)
  asset: asset id
}
Arguments:
  • res (object) – response

Function: get_blastdata

Determine filter details, like number of hit results. REST

GET /service/exec/set_filter data: eg. {asset: ‘151_1517462263883’, dataset: ‘sample_data/json/volvox’}

Return data: eg. { result: ‘success’, hits: 792, filteredHits: 501 }

get_blastdata(req, res)
Arguments:
  • req (object) – request
  • res (object) – response

Function: get_trackdata

Fetch the GFF3 file of the prior filter operation

GET /service/exec/set_filter

get_trackdata(req, res)
Arguments:
  • req (type) – request
  • res (type) – response

Module: services/galaxyService

Description

This job service is functionally equivelant to basicWorkflowService, however, NCBI operations are sent through galaxy workflow for processing.

Job submission example:

var postData = {
      service: "jblast",
      dataset: "sample_data/json/volvox",
      region: ">ctgA ctgA:44705..47713 (- strand) class=remark length=3009\nacatccaatggcgaacataa...gcgagttt",
      workflow: "NCBI.blast.workflow.js"
  };
$.post( "/job/submit", postData , function( result ) {
    console.log( result );
}, "json");

Configuration:

 // Galaxy settings
 galaxy: {
     // Galaxy API path
     galaxyUrl: "http://localhost:8080",

     // Galaxy installation path
     galaxyPath: "/var/www/html/galaxy",

     // Galaxy API key (you must obtain this from your Galaxy installation)
     galaxyAPIKey: "c7be32db9329841598b1a5705655f633",

     // The default Galaxy History where workflows will execute
     historyName: "Unnamed history"
 },

 jblast: {
    // The subdir where blast results will be deposited (i.e. ``sample_data/json/volvox/jblastdata``)
    blastResultPath: "jblastdata",

    // The category for successful blast results in the track selector
    blastResultCategory: "JBlast Results",

    // Track template of the blast result track that will be inserted in trackList.json
    trackTemplate: "jblastTrackTemplate.json",

    // Type of file that will be imported for processing blast.
    import: ["blastxml"],


    // BLAST profiles
    // blast profiles are parameter lists that translate to blastn cli parameters sets
    // (i.e. for "remote_htgs" would translate to "blastn -db htgs -remote")
    // These will override any default parameters defined in ``blastjs``
    //
    // Blast profiles generally apply to basicWorkflowService only
    // and do no apply to galaxyService.
    //
    // Our example uses a subset of htgs, an NCBI curated blast database.
    // So, it is our default profile.
    defaultBlastProfile: 'htgs',
    blastProfiles: {
        'htgs': {
            'db': 'htgs'
        },
        'remote_htgs': {
            'db': 'htgs',
            'remote': ""
        }
    }
},
// list of services that will get registered.
services: {
    'galaxyService':          {name: 'galaxyService',         type: 'workflow', alias: "jblast"},
    'filterService':            {name: 'filterService',         type: 'service'},
    'entrezService':            {name: 'entrezService',         type: 'service'}
},

Function: validateParams

job service validation

validateParams(params)
Arguments:
  • params (object) – parameters
Return val:

0 if successful, otherwise failure

Function: generateName

job service generate name

generateName(params)
Arguments:
  • params (object) – parameters
Return string:

name of job

Function: beginProcessing

job service begin

beginProcessing(kJob)
Arguments:
  • kJob (object) – kue job object

Function: get_workflows

get_workflows()

Function: get_hit_details

get_hit_details()

Module: services/galaxyUtils

Description

This provides functional support to galaxyService job service.

Function: init

Initialize module

init(cb, cberr)
Arguments:
  • cb (type) – Initialize module
  • cberr (type) – Initialize module
Return undefined:
 

Initialize module

Function: galaxyGetPromise

galaxyGetPromise()

Function: galaxyPostPromise

galaxyPostPromise()

Function: galaxyGET

send JSON GET request to galaxy server

galaxyGET(api, cb)
Arguments:
  • api (type) – i.e. ‘/api/histories’
  • cb (type) – callback i.e. function(retval)

Function: getHistoryId

getHistoryId()
Return string:history id

Function: getHistoryName

getHistoryName()
Return string:history name

Function: initHistory

acquire history id from galaxy

initHistory(cb)
Arguments:
  • cb (type) – acquire history id from galaxy

Function: getWorkflows

get workflows

getWorkflows(cb)
Arguments:
  • cb (type) – get workflows
Return undefined:
 

get workflows

Function: sendFile

send file to galaxy

sendFile(theFile, hId, cb, cberr)
Arguments:
  • theFile (type) – send file to galaxy
  • hId (type) – send file to galaxy
  • cb (type) – send file to galaxy
  • cberr (type) – send file to galaxy
Return undefined:
 

send file to galaxy

Function: beginProcessing

Job service, job entry point.

beginProcessing(kJob)
Arguments:
  • kJob (object) – reference to kue job object

Function: beginProcessing2

beginProcessing2()

Function: monitorWorkflow

Monitor workflow and exit upon completion of the workflow

monitorWorkflow(kJob)
Arguments:
  • kJob (object) – Monitor workflow and exit upon completion of the workflow

Function: doCompleteAction

Read output of last generated file, copy results to /jblastdata, insert track to trackList.json.

doCompleteAction(kJob, hista)
Arguments:
  • kJob (object) – kue job object
  • hista (object) – associative array of histories

Module: services/jblastPostAction

Description

This module implements the actions that occur after a galaxy workflow completes. It supports galaxyService job service.

Function: postMoveResultFiles

postMoveResultFiles()

Function: processFilter

processFilter()

Function: postMoveResultFiles

this generates track template

postMoveResultFiles(kJob, cb)
Arguments:
  • kJob (type) – kue job object
  • cb (type) – callback

Function: processFilter

Generate the GFF file

processFilter(kJob, newTrackJson, cb)
Arguments:
  • kJob (type) – = kue job object
  • newTrackJson (type) – working track object
  • cb (type) – callback

Function: getHits

return number of hits

getHits(kJob, newTrackJson)
Arguments:
  • kJob (object) – kue job object
  • newTrackJson (JSON) – working track object
Return Number:

number of hits

Member: requestp:

Member: path:

Member: Promise:

Member: fs:

Member: deferred:

Member: filter:

Member: offsetfix:

Member: blast2json:

Member: galaxy:

Member: _:

Member: newTrackJson:

Module: services/offsetfix

Local Navigation

Description

This module fixes the offsets of blast search results.