Theme:
v
APL+Win 13.2 ⎕CSE Tutorial (3)

This page contains more examples showing how to use the new APL+Win Interface to .Net 4.5 using the new ⎕CSE System Function

Example 9: Reading the content of a Web page

In this example we will create a simple script to programmatically retrieve the content of any Internet Web page.

Here is the function:

    ∇ R←ReadWebPage url;s;z;⎕cself
[1]   ⍝∇ R←ReadWebPage url -- Programmatically retrieve the content of any Internet Web page
[2]
[3]   ⎕cself←'c'⎕cse'Init' 'System' 'System.Net'
[4]
[5]   s←''
[6]   s←s,⊂'using System.Text;'
[7]   s←s,⊂'using System.Net;'
[8]   s←s,⊂'using (WebClient client = new WebClient())'
[9]   s←s,⊂'{'
[10]  s←s,⊂'    byte[] bytes = client.DownloadData("',url,'");'
[11]  s←s,⊂'    string result = Encoding.UTF8.GetString(bytes);'
[12]  s←s,⊂'}'
[13]  s←⊃s
[14]
[15]  z←⎕cse'Exec's
[16]  :if z=¯1 ⋄ ⎕error⎕cse'GetLastError' ⋄ :endif
[17]
[18]  R←⎕cse'GetValue' 'result'
    ∇

Here are some comments:

  • first we create an isntance of the CSE object on line 3, declaring we will be using the System.Net DLL
  • then, on lines 6 and 7 we declare we will be using the System.Text namespace and the System.Net namespace
  • then we create a block of code with a using statement, creating an instance of the WebClient class in the using statement: thi s ensures that the client instance of WebClient will be disposed of automatically when the block of code finishes executing (i.e. when the next } is reached)
  • on line 10 we simply call the DownloadData method of the client object to retrieve the content of the Web page defined by the url APL argument: however, since DownloadData returns a vector of bytes (i.e. byte[] in C#) we need to declare the result as being a byte[] variable
  • since APL does not know what a byte[] vector is, we need to concert this byte[] vector into a string: thi is easily done using the GeString method of the Encoding.UTF8 class on line 11
  • on line 15 we execute the script, therefore creating the variable result
  • finally on line 18 all what's left to do is to read the content of the .Net result variable

So, let's try to run function ReadWebPage:

      aaa←ReadWebPage'http://www.google.com'
CSE ERROR: (1,1): error CS0103: The name 'result' does not exist in the current context
ReadWebPage[19] R←⎕cse'GetValue' 'result'
                  ^

We get an error. I did that intentionally to teach one thing to novice C# users: when you declare a variable inside a block of code (i.e. inside code surrounded b y curly braces), the variable is local to this block of code; therefore it disappears as the block of code finishes executing

To solve the problem one simply needs to declare the result variable outside the block of code as in:

    ∇ R←ReadWebPage url;s;z;⎕cself
[1]   ⍝∇ R←ReadWebPage url -- Programmatically retrieve the content of any Internet Web page
[2]
[3]   ⎕cself←'c'⎕cse'Init' 'System' 'System.Net'
[4]
[5]   s←''
[6]   s←s,⊂'using System.Text;'
[7]   s←s,⊂'using System.Net;'
[8]   s←s,⊂'string result;'
[9]   s←s,⊂'using (WebClient client = new WebClient())'
[10]  s←s,⊂'{'
[11]  s←s,⊂'    byte[] bytes = client.DownloadData("',url,'");'
[12]  s←s,⊂'    result = Encoding.UTF8.GetString(bytes);'
[13]  s←s,⊂'}'
[14]  s←⊃s
[15]
[16]  z←⎕cse'Exec's
[17]  :if z=¯1 ⋄ ⎕error⎕cse'GetLastError' ⋄ :endif
[18]
[19]  R←⎕cse'GetValue' 'result'
    ∇

Now that we've fixed the error in our script, we are able to read the content of any Web page using function ReadWebPage:

      aaa←ReadWebPage'http://www.google.com'

      ⍴aaa
43604

      ⎕pw←80

      1000↑aaa~⎕tclf
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage"><head><me
      ta itemprop="image" content="/images/google_favicon_128.png"><title>Google
      </title><script>(function(){window.google={kEI:"jJY1Ut-DMsfFswbU-IDgCQ",ge
      tEI:function(a){for(var b;a&&(!a.getAttribute||!(b=a.getAttribute("eid")))
      ;)a=a.parentNode;return b||google.kEI},https:function(){return"https:"==wi
      ndow.location.protocol},kEXPI:"17259,4000116,4002855,4004949,4005610,40067
      27,4007080,4007231,4007278,4007463,4007661,4007714,4008067,4008133,4008142
      ,4008488,4008567,4008725,4008750,4008821,4009002,4009021,4009057,4009220,4
      009353,4009386,4009476,4009565,4009597,4009812,4009822,4009873,4010015,401
      0057,4010061,4010066,4010131,4010184,4010291,4010307",kCSI:{e:"17259,40001
      16,4002855,4004949,4005610,4006727,4007080,4007231,4007278,4007463,4007661
      ,4007714,4008067,4008133,4008142,4008488,4008567,4008725,4008750,4008821,4
      009002,4009021,4009057,4009220,4009353,4009386,4009476,4009565,4009597,400
      9812,4009822,4009873,4010015,401

Note that the script could be written in a shorter way, as follows:

    ∇ R←ReadWebPage url;s;z;⎕cself
[1]   ⍝∇ R←ReadWebPage url -- Programmatically retrieve the content of any Internet Web page
[2]
[3]   ⎕cself←'c'⎕cse'Init' 'System' 'System.Net'
[4]
[5]   s←''
[6]   s←s,⊂'using System.Text;'
[7]   s←s,⊂'using System.Net;'
[8]   s←s,⊂'string result;'
[9]   s←s,⊂'using (WebClient client = new WebClient())'
[10]  s←s,⊂'    result = Encoding.UTF8.GetString(client.DownloadData("',url,'"));'
[11]  s←⊃s
[12]
[13]  z←⎕cse'Exec's
[14]  :if z=¯1 ⋄ ⎕error⎕cse'GetLastError' ⋄ :endif
[15]
[16]  R←⎕cse'GetValue' 'result'
    ∇

Note that we have combined lines 11 and 12 of the previous script into line 10 of this script and then, because the using block of code is now reduced to only one line, we could remove the curly braces (they are optional when a block of C# code is only one line long).

Our ReadWebPage APL function still works perfectly well:

      aaa←ReadWebPage'http://www.devexpress.com'

      ⍴aaa
118473

      1000↑aaa~⎕tclf

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head><link rel="stylesheet" type="text/css" href="/DXR.axd?r=2_1,1_4,2_2,1_3,1_2-LqE77" /><script type="text/javascript">
// <![CDATA[
var IsLoggedIn = false; var IsEmployee = false; var IsPostback = false; var PreferredLanguage = 'en-US'
// ]]>
</script><script type="text/javascript">
// <![CDATA[
var _wsData = { "Settings": { "TrackingEnabled": true }, "UserInfo": { "IsEmployee": false, "IsLoggedIn": false } };
//]]>
</script><script type="text/javascript" src="/DXR.axd?r=9999_23-EOvq7"></script><link href="App_Themes/DX/Common/common.css?wsCache=2_0_1_17633" type="text/css" rel="stylesheet" /><link href="App_Themes/DX/Co
      mmon/core.css?wsCache=2_0_1_17633" type="text/css" rel="stylesheet" /><link href="App_Themes/DX/Common/visual.css?wsCache=2_0_1_17633" type="text/css" rel="stylesheet" /><link href="App_Them

The above example shows how you can write something extremely useful (and otherwise difficult to develop in APL) in just a few lines of C# code: this is typical and there are countless useful things you can do this way using C#.

Of course, the recommended way is to develop these C# utilities using Visual Studio and to group them all into one or more DLLs.

Example 10: Extracting data from an XML file

In this example, we want to be able to extract any kind of data from an XML file.

For this example we will be using the sample Books.xml file provided by Microsoft (see: http://msdn.microsoft.com/en-us/library/ms762271(v=vs.85).aspx) which reads:


<catalog>
   <book id="bk101">
      <author>Gambardella, Matthew</author>
      <title>XML Developer's Guide</title>
      <genre>Computer</genre>
      <price>44.95</price>
      <publish_date>2000-10-01</publish_date>
      <description>An in-depth look at creating applications 
      with XML.</description>
   </book>
   <book id="bk102">
      <author>Ralls, Kim</author>
      <title>Midnight Rain</title>
      <genre>Fantasy</genre>
      <price>5.95</price>
      <publish_date>2000-12-16</publish_date>
      <description>A former architect battles corporate zombies, 
      an evil sorceress, and her own childhood to become queen 
      of the world.</description>
   </book>
   <book id="bk103">
      <author>Corets, Eva</author>
      <title>Maeve Ascendant</title>
      <genre>Fantasy</genre>
      <price>5.95</price>
      <publish_date>2000-11-17</publish_date>
      <description>After the collapse of a nanotechnology 
      society in England, the young survivors lay the 
      foundation for a new society.</description>
   </book>
   <book id="bk104">
      <author>Corets, Eva</author>
      <title>Oberon's Legacy</title>
      <genre>Fantasy</genre>
      <price>5.95</price>
      <publish_date>2001-03-10</publish_date>
      <description>In post-apocalypse England, the mysterious 
      agent known only as Oberon helps to create a new life 
      for the inhabitants of London. Sequel to Maeve 
      Ascendant.</description>
   </book>
   <book id="bk105">
      <author>Corets, Eva</author>
      <title>The Sundered Grail</title>
      <genre>Fantasy</genre>
      <price>5.95</price>
      <publish_date>2001-09-10</publish_date>
      <description>The two daughters of Maeve, half-sisters, 
      battle one another for control of England. Sequel to 
      Oberon's Legacy.</description>
   </book>
   <book id="bk106">
      <author>Randall, Cynthia</author>
      <title>Lover Birds</title>
      <genre>Romance</genre>
      <price>4.95</price>
      <publish_date>2000-09-02</publish_date>
      <description>When Carla meets Paul at an ornithology 
      conference, tempers fly as feathers get ruffled.</description>
   </book>
   <book id="bk107">
      <author>Thurman, Paula</author>
      <title>Splish Splash</title>
      <genre>Romance</genre>
      <price>4.95</price>
      <publish_date>2000-11-02</publish_date>
      <description>A deep sea diver finds true love twenty 
      thousand leagues beneath the sea.</description>
   </book>
   <book id="bk108">
      <author>Knorr, Stefan</author>
      <title>Creepy Crawlies</title>
      <genre>Horror</genre>
      <price>4.95</price>
      <publish_date>2000-12-06</publish_date>
      <description>An anthology of horror stories about roaches,
      centipedes, scorpions  and other insects.</description>
   </book>
   <book id="bk109">
      <author>Kress, Peter</author>
      <title>Paradox Lost</title>
      <genre>Science Fiction</genre>
      <price>6.95</price>
      <publish_date>2000-11-02</publish_date>
      <description>After an inadvertant trip through a Heisenberg
      Uncertainty Device, James Salway discovers the problems 
      of being quantum.</description>
   </book>
   <book id="bk110">
      <author>O'Brien, Tim</author>
      <title>Microsoft .NET: The Programming Bible</title>
      <genre>Computer</genre>
      <price>36.95</price>
      <publish_date>2000-12-09</publish_date>
      <description>Microsoft's .NET initiative is explored in 
      detail in this deep programmer's reference.</description>
   </book>
   <book id="bk111">
      <author>O'Brien, Tim</author>
      <title>MSXML3: A Comprehensive Guide</title>
      <genre>Computer</genre>
      <price>36.95</price>
      <publish_date>2000-12-01</publish_date>
      <description>The Microsoft MSXML3 parser is covered in 
      detail, with attention to XML DOM interfaces, XSLT processing, 
      SAX and more.</description>
   </book>
   <book id="bk112">
      <author>Galos, Mike</author>
      <title>Visual Studio 7: A Comprehensive Guide</title>
      <genre>Computer</genre>
      <price>49.95</price>
      <publish_date>2001-04-16</publish_date>
      <description>Microsoft Visual Studio 7 is explored in depth,
      looking at how Visual Basic, Visual C++, C#, and ASP+ are 
      integrated into a comprehensive development 
      environment.</description>
   </book>
</catalog>

We can write the following simple APL function to extract any data from the XML file:

    ∇ R←file ReadXmlData xpath;s;z;⎕cself
[1]   ⍝∇ R←file ReadXmlData xpath -- Extract XML data from an XML file
[2]
[3]   ⎕cself←'c'⎕cse'Init' 'System' 'System.Xml'
[4]
[5]   s←''
[6]   s←s,⊂'using System.Collections.Generic;'
[7]   s←s,⊂'using System.Xml;'
[8]   s←s,⊂'XmlDocument doc = new XmlDocument();'
[9]   s←s,⊂'doc.Load(@"',file,'");'
[10]  s←s,⊂'XmlNodeList nodes = doc.SelectNodes("',xpath,'");'
[11]  s←s,⊂'List<string> data = new List<string>();'
[12]  s←s,⊂'foreach (XmlNode node in nodes)'
[13]  s←s,⊂'   data.Add(node.InnerText);'
[14]  s←s,⊂'string[] result = data.ToArray();'
[15]  s←⊃s
[16]
[17]  z←⎕cse'Exec's
[18]  :if z=¯1 ⋄ ⎕error⎕cse'GetLastError' ⋄ :endif
[19]
[20]  R←⎕cse'GetValue' 'result'
    ∇

Here are some comments:

  • on line 3 we create an instance of the CSE object declaring we will be using the System.Xml .Net Framework DLL
  • on line 6 and 7 we declare we will be using classes in the System.Collections.Generic and in the System.Xml namespaces
  • on line 8 we create an instance of the XmlDocument object to allow us to read our XML file
  • on line 9 we read our entire Books.xml file into memory using the Load method of the doc XmlDocument instance
  • on line 10 we run our xpath query by passing it as an argument to the SelectNodes method: the SelectNodes method returns a collection of XML nodes which is an XmlNodeList object
  • on line 11 we create an instance of a generic List of strings: this creates an empty collection which can only contain strings
  • once you have a collection in C#, it is easy to traverse the collection using a foreach loop: so we loop on the nodes we have extacted from the XML file on line 10 and for each node, we extract its InnerText property and append it to our List of strings called data
  • after the foreach loop, because APL cannot understand collections which are C# specific data types, we need to convert the List of strings collection (List<string>) to an array of strings (string[])

We can extract data from the XML file as follows, running the abov script:

      'd:\aplwin\ele\books.xml'ReadXmlData'/catalog/book/price'     
 44.95 5.95 5.95 5.95 5.95 4.95 4.95 4.95 6.95 36.95 36.95 49.95

      ⊃'d:\aplwin\ele\books.xml'ReadXmlData'/catalog/book/author'     
Gambardella, Matthew
Ralls, Kim
Corets, Eva
Corets, Eva
Corets, Eva
Randall, Cynthia
Thurman, Paula
Knorr, Stefan
Kress, Peter
O'Brien, Tim
O'Brien, Tim
Galos, Mike

      'd:\aplwin\ele\books.xml'ReadXmlData'/catalog/book[price>6]/price'     
 44.95 6.95 36.95 36.95 49.95

      'd:\aplwin\ele\books.xml'ReadXmlData'//book[price>6]/price'     
 44.95 6.95 36.95 36.95 49.95

      "d:\aplwin\ele\books.xml"ReadXmlData"//book/@id"      
 bk101 bk102 bk103 bk104 bk105 bk106 bk107 bk108 bk109 bk110 bk111 bk112

      (((⍴aaa)÷3),3)⍴aaa←"d:\aplwin\ele\books.xml"ReadXmlData"//author | //title | //price"       
 Gambardella, Matthew XML Developer's Guide                  44.95
 Ralls, Kim           Midnight Rain                          5.95
 Corets, Eva          Maeve Ascendant                        5.95
 Corets, Eva          Oberon's Legacy                        5.95
 Corets, Eva          The Sundered Grail                     5.95
 Randall, Cynthia     Lover Birds                            4.95
 Thurman, Paula       Splish Splash                          4.95
 Knorr, Stefan        Creepy Crawlies                        4.95
 Kress, Peter         Paradox Lost                           6.95
 O'Brien, Tim         Microsoft .NET: The Programming Bible  36.95
 O'Brien, Tim         MSXML3: A Comprehensive Guide          36.95
 Galos, Mike          Visual Studio 7: A Comprehensive Guide 49.95

      ]display (((⍴aaa)÷3),3)⍴aaa
.→--------------------------------------------------------------------.
↓.→-------------------..→--------------------.                 .→----.∣
∣∣Gambardella, Matthew∣∣XML Developer's Guide∣                 ∣44.95∣∣
∣'--------------------''---------------------'                 '-----'∣
∣.→---------.          .→------------.                         .→---. ∣
∣∣Ralls, Kim∣          ∣Midnight Rain∣                         ∣5.95∣ ∣
∣'----------'          '-------------'                         '----' ∣
∣.→----------.         .→--------------.                       .→---. ∣
∣∣Corets, Eva∣         ∣Maeve Ascendant∣                       ∣5.95∣ ∣
∣'-----------'         '---------------'                       '----' ∣
∣.→----------.         .→--------------.                       .→---. ∣
∣∣Corets, Eva∣         ∣Oberon's Legacy∣                       ∣5.95∣ ∣
∣'-----------'         '---------------'                       '----' ∣
∣.→----------.         .→-----------------.                    .→---. ∣
∣∣Corets, Eva∣         ∣The Sundered Grail∣                    ∣5.95∣ ∣
∣'-----------'         '------------------'                    '----' ∣
∣.→---------------.    .→----------.                           .→---. ∣
∣∣Randall, Cynthia∣    ∣Lover Birds∣                           ∣4.95∣ ∣
∣'----------------'    '-----------'                           '----' ∣
∣.→-------------.      .→------------.                         .→---. ∣
∣∣Thurman, Paula∣      ∣Splish Splash∣                         ∣4.95∣ ∣
∣'--------------'      '-------------'                         '----' ∣
∣.→------------.       .→--------------.                       .→---. ∣
∣∣Knorr, Stefan∣       ∣Creepy Crawlies∣                       ∣4.95∣ ∣
∣'-------------'       '---------------'                       '----' ∣
∣.→-----------.        .→-----------.                          .→---. ∣
∣∣Kress, Peter∣        ∣Paradox Lost∣                          ∣6.95∣ ∣
∣'------------'        '------------'                          '----' ∣
∣.→-----------.        .→------------------------------------. .→----.∣
∣∣O'Brien, Tim∣        ∣Microsoft .NET: The Programming Bible∣ ∣36.95∣∣
∣'------------'        '-------------------------------------' '-----'∣
∣.→-----------.        .→----------------------------.         .→----.∣
∣∣O'Brien, Tim∣        ∣MSXML3: A Comprehensive Guide∣         ∣36.95∣∣
∣'------------'        '-----------------------------'         '-----'∣
∣.→----------.         .→-------------------------------------..→----.∣
∣∣Galos, Mike∣         ∣Visual Studio 7: A Comprehensive Guide∣∣49.95∣∣
∣'-----------'         '--------------------------------------''-----'∣
'∊--------------------------------------------------------------------'

Note that the XPath language which we have been using in this example (the ReadXmlData right argument must be a valid XPath query) is very powerful and really easy to use.

Here are sample XPath queries:

/catalog/book/author select the author of all the books in the catalog element
/catalog/book[price<5] select all the books with a price less than 5
/catalog/book[2] select the second book in the XML file
/catalog/book[3]/title select the title of the third book in the XML file
//title select all the title elements in the XML file
/catalog/book[last()]/genre select the genre of the last book in the XML file
/*/*/genre select all the genre elements that have 2 ancestors
//author | //title select all the authors and all the titles in the document
//* select all elements in the document
//@id select all the id attributes in the document
//book[@id] select all book elements that have an id attribute in the document
//book[@*] select all book elements that have any attribute in the document
//book[@id='bk108'] select all book elements that have an id attribute equal to 'bk108' in the document

Once again we have been able to perform a complex task in just a few lines of simple code using C#.

 

CSE Tutorial (1)...
CSE Tutorial (2)...
CSE Tutorial (4)...
CSE Tutorial (5)...