JDN API

This document provides a guide on how to create a list of JDN locators for a webpage using JDN API endpoints.

List of sources:

Backend API docs in Swagger
Backend API docs in ReDoc (same information as in Swagger, just another form of representation)
Backend WebSocket API documentation
Backend GitHub: https://github.com/jdi-testing/jdi-qasp-ml
Frontend GitHub: https://github.com/jdi-testing/jdn-ai/

Scenario: Locators List for HTML5 Webpage

Prepare the DOM Tree Document: Create a transformed version of the DOM tree.
you could use the reference code part: create a transformed DOM tree
Add JDN-hashes: Assign a unique 'jdn-hash' ID to every element in the DOM tree for later mapping.
You can refer to the code snippet : jdn-hash-id
Request HTML5 Predict API with the following parameters:
1. Transformed and prepared DOM tree document with jdn-hash attributes set for elements. Ideally, all of the page styles and JS code should be included into the document to achieve the best possible results when determining elements visibility.
2. List of elements with some additional information that you’d like to be analyzed by a model. Must contain at least one element.
3. Viewport dimensions (optional). Could be supplied to get better results with elements visibility calculation.
  You can find more information via link

Request:

{

    "document": "<html lang=\"en\" jdn-hash=\"111111\"><body id=\"body\" jdn-hash=\"222222\"><h1 jdn-hash=\"333333\">Hello</h1></body></html>",
    "elements": "[{\"tag_name\":\"BODY\",\"element_id\":\"222222\",\"parent_id\":\"111111\",\"x\":0,\"y\":0,\"width\":1070,\"height\":969,\"displayed\":true,\"onmouseover\":null,\"onmouseenter\":null,\"attributes\":{\"id\": \"body\", \"jdn-hash\":\"222222\"},\"text\": \"Hello\"}]",
    "viewport": "{\"width\": 1920,\"height\": 1080}"
}

JDN - hash is unique for the session (<body jdn-hash="5487869680421457123776496688">)

Response:

[
    {
        "element_id": "9107226170421457128369704519", // JDN-hash
        "predicted_label": "textfield", // generated by frontend
        "childs": null,
        "displayed": false, 
        "is_shown": false // hidden element
    },
    {
        "element_id": "6129314823421457123001427800",
        "predicted_label": "textfield",
        "childs": [
            "1347254450421457129229630841"
        ],
        "displayed": true,
        "is_shown": true
    },
    {
        "element_id": "2638973673421457120564469341",
        "predicted_label": "textfield",
        "childs": null,
        "displayed": true,
        "is_shown": true
    },
    {
        "element_id": "3535110229421457123259229507",
        "predicted_label": "textfield",
        "childs": null,
        "displayed": true,
        "is_shown": false
    },
    {
        "element_id": "9785700149421457120869751842",
        "predicted_label": "textarea",
        "childs": null,
        "displayed": true,
        "is_shown": true
    },
    {
        "element_id": "1207655779421457128652021367",
        "predicted_label": "textfield",
        "childs": null,
        "displayed": true,
        "is_shown": false
    },
    {
        "element_id": "8579850601421457124109026854",
        "predicted_label": "textfield",
        "childs": [
            "2283749025421457126077190380"
        ],
        "displayed": true,
        "is_shown": true
    },
    {
        "element_id": "6751662626421457121898869568",
        "predicted_label": "textfield",
        "childs": null,
        "displayed": true,
        "is_shown": true
    },
    {
        "element_id": "6545460502421457124434401870",
        "predicted_label": "table",
        "childs": [
            "5409933759421457124216081574",
            "9955593493421457125874174848",
            "7327528983421457122170103911",
            "6117317753421457124179824293",
            "5100048133421457120794692029",
            "3069649749421457127661098981",
            "7931790657421457125819134246"
        ],
        "displayed": true,
        "is_shown": true
    },
    {
        "element_id": "5409933759421457124216081574",
        "predicted_label": "button",
        "childs": null,
        "displayed": true,
        "is_shown": true
    },
    {
        "element_id": "7327528983421457122170103911",
        "predicted_label": "button",
        "childs": null,
        "displayed": true,
        "is_shown": true
    },
    {
        "element_id": "5100048133421457120794692029",
        "predicted_label": "button",
        "childs": null,
        "displayed": true,
        "is_shown": true
    },
    {
        "element_id": "7931790657421457125819134246",
        "predicted_label": "button",
        "childs": null,
        "displayed": true,
        "is_shown": true
    }
]

Remove Hidden Elements: Eliminate elements from the document where 'is_shown' is false.
reference the code part: remove hidden elements
Start Locators Generation: Connect to the WebSocket and follow the API described in the WebSocket documentation

Additional considerations:

Authentication: without authorization
MUI Predict http://78.47.220.208:5001/redoc#operation/mui_predict_mui_predict_post
Angular Predict http://78.47.220.208:5001/redoc#operation/angular_predict_angular_predict_post