C#使用Xpath进行html解析

0x00.Xpath

XPath 使用路径表达式来选取 XML 文档中的节点或者节点集。这些路径表达式和我们在常规的电脑文件系统中看到的表达式非常相似。

XPath 含有超过 100 个内建的函数。这些函数用于字符串值、数值、日期和时间比较、节点和 QName 处理、序列处理、逻辑值等等。


0x01.添加引用

Nuget中下载 HtmlAgilityPack,项目中引用

using HtmlAgilityPack;

0x02.Load html页面

HtmlDocument index = web.Load("http://www.sonystyle.com.cn/mysony/acafe/index.htm");

0x03.查看所需元素的Xpath

浏览器以火狐为例

打开所需网页,F12进入控制台

定位到所需元素,右键→复制→Xpath

e.g:/html/body/div[3]/div[1]/div/div/div[1]/div[1]/div[1]


0x04.DocumentNode.SelectNodes的用法

DocumentNode.SelectNodes选择的是多个节点,需要使用集合

private List<string>list;

foreach (var sth in index.DocumentNode.SelectNodes("//ul[@class='slides']/li/a/img"))
{
  list.Add(base_url + index_img.GetAttributeValue("src", ""));//获取对应元素的src值
}

//*是固定格式,*表示匹配中的元素
/ 表示下一次
@表示对应的属性,如//ul[@class=’slides’]表示class为slides的ul元素


0x05.DocumentNode.SelectNodes的用法

DocumentNode.SelectNodes选择的是单一元素,所以需要使用该元素的绝对路径

var sth = index.DocumentNode.SelectSingleNode("//div/li/a/img");
var attribute=sth.GetAttributeValue("src", ""));

0x06.更多

Xpath参考手册

UWP模拟获取验证码

0x00.添加引用

using System.Threading.Tasks;
using System.Net.Http;

0x01.创建新Task

public static Task<HttpResponseMessage> VerCode(string Req)//Req为获取验证码的请求连接,可去需要的网站分析请求获取
{
    return Task.Run(() =>
    {
        HttpClient request = new HttpClient(HttpClientHandler);
        HttpResponseMessage response;
        {
            request.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0");
            request.DefaultRequestHeaders.Add("Accept", "*/*");
            request.DefaultRequestHeaders.Add("Accept-Language", "zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2");
            request.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate, br");
        }//模拟请求头
        try
        {
            response = request.GetAsync(Req).Result;//响应内容为MIME
        }
        catch
        {
            response = null;
        }
        return response;
    });
}

0x02.流形式读取

private async Task GetVerCodeAsync(string Req)//Req为获取验证码的请求连接,可去需要的网站分析请求获取
{
    HttpResponseMessage response = await Sign.VerCode(Req);
    byte[] streamByte = response.Content.ReadAsByteArrayAsync().Result;
    MemoryStream ms = new MemoryStream(streamByte);
    BitmapImage img = new BitmapImage();
    await img.SetSourceAsync(ms.AsRandomAccessStream());//img为Image控件
    VerCodeImg.Source = img;
}

0x03.点击刷新(因网站而异)

private async void VerCodeBtn_Click(object sender, RoutedEventArgs e)
{
    a = a + 1;
    string plus = "?test=" + a;
    string baseReq = "https://www.sonystyle.com.cn/mysony/campaign/api/captcha.do" + plus;
    await this.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, async () =>
    {
        await GetVerCodeAsync(baseReq);
    });
}

0x04.相关

MIME 参考手册

UWP标题栏扩展

0x00.添加引用

在默认引用下添加

using Windows.ApplicationModel.Core

0x01.实现

public MainPage()
{
  this.InitiallizeComponent();
  var titlebar=CoreApplication.GetCurrentView().TitleBar;
  titlebar.ExtendViewIntoTitleBar=true;
  Window.Current.SetTitleBar(thingsuneed);//thingsuneed 为需要扩展至标题栏的控件的X:name
}

 

 

UWP 使用BackgroundTransfer下载图片

 0x00.添加引用

using System;
using System.Threading.Tasks;
using Windows.Networking.BackgroundTransfer;
using Windows.Storage;

0x01.下载

Uri uri = new Uri(imageUri);//需要下载图片的Uri,exp:http*****.jpg
BackgroundDownloader backgroundDownload = new BackgroundDownloader();
StorageFolder folder = await KnownFolders.PicturesLibrary.CreateFolderAsync("Acafe", CreationCollisionOption.OpenIfExists);//此处为下载路径,可是要Openfolder改为自定义路径
StorageFile newFile = await folder.CreateFileAsync(imageName, CreationCollisionOption.OpenIfExists);//创建文件,需要定义文件名
DownloadOperation download = backgroundDownload.CreateDownload(uri, newFile);//创建下载器
await download.StartAsync();//启动

 

UWP 自定义Popup控件的自适应

0x00.前提

自定义了一个查看图片细节的控件,本质是一个Image控件,但是应为采用了左侧为NavigationView右侧为Frame的导航模式,为了使该控件能覆盖到左侧的NavigationView之上,只能采用Popup。

这也导致该控件的容器不能是Frame/Page中的Grid,从而不能直接随窗体一起拉伸,可以采用SizeChanged事件和INotifyPropertyChanged接口来实现,也可使使用MeasurePopupSize()


0x01.实现

public Image_Detail()
{
    this.InitializeComponent();
    _popup = new Popup();
    ApplicationView.GetForCurrentView().VisibleBoundsChanged += (s, e) =>
    {
        MeasurePopupSize();
    };
    MeasurePopupSize();
    _popup.Child = this;
}

private void MeasurePopupSize()
{
    this.Width = ApplicationView.GetForCurrentView().VisibleBounds.Width;
    this.Height = ApplicationView.GetForCurrentView().VisibleBounds.Height;
}

C# Json反序列化

0x00.使用工具将Json格式化

Json作为一种数据交换格式,具有简洁和清晰的结构。
在线Json格式化校验工具:http://tool.chinaz.com/tools/jsonformat.aspx

在线转换C#类工具:http://json2csharp.chahuo.com/

Json数据
{
    "code": "00",
    "count": 1,
    "followersList": [
        {
            "followerDate": "2017-11-29 11:11:10",
            "follower_id": "WEP2DU",
            "userIconPath": "http://www.sonystyle.com.cn/mysony/acafe/myspace/images/space/default.gif"
        }
    ],
    "programId": 1,
    "userId": "YIOMOOOO"
}
使用C#类工具转换后新建一个”JsonFollower”类
class JsonFollower
{
    public class FollowersList
    {
        public string followerDate { get; set; }
        public string follower_id { get; set; }
        public string userIconPath { get; set; }
    }

    public class RootObject
    {
        public string code { get; set; }
        public string count { get; set; }
        public List<followerslist> followersList { get; set; }
        public List<followering> following { get; set; }
        public string programId { get; set; }
        public string userId { get; set; }
    }
}

0x01.添加引用

using System.Collections.Generic;
using System.Net.Http;
using Newtonsoft.Json;//在Nuget中下载对应类库

0x02.模拟发送请求获取Json

HttpClient request = new HttpClient();
HttpResponseMessage MyFolResponse = request.GetAsync("http://www.sonystyle.com.cn/mysony/campaign/api/program.do?methodName=getMyFollowersList&programId=1" + "&userId=YIOMOOOO").Result;
string a = MyFolResponse.Content.ReadAsStringAsync().Result;
JsonFollower.RootObject rb1 = JsonConvert.DeserializeObject<JsonFollower.RootObject>(a);

0x03.遍历RootObject并转化为对象

List<FollowerInfo> FolInfo = new List<FollowerInfo >();
List<string> followerDate = new List<string>();
List<string> follower_id = new List<string>();

foreach (JsonFollower.FollowersList fl in rb1.followersList)
{
    followerDate.Add(fl.followerDate);
    follower_id.Add(fl.follower_id);
}

for (int i = 0; i < follower_id.Count; i++)
{
    FollowerInfo FI = new FollowerInfo();
    FI.followerDate = followerDate[i];
    FI.follower_id = follower_id[i];
    FolInfo.Add(FI);
}
return FolInfo;

Jumony.Core爬虫学习 [1]

 0x00.Nuget 中下载 Jumony.Core 安装至当前项目

0x01.获取http://www.sonystyle.com.cn/mysony/acafe/index.htm 首页滚动图片URL

foreach (var img in new JumonyParser().LoadDocument("http://www.sonystyle.com.cn/mysony/acafe/index.htm/",Encoding.GetEncoding(65001)).Find("[target=_blank]>img[src*=\"index\"]"))<br> textBox.Text = textBox .Text + "http://www.sonystyle.com.cn/mysony/acafe/" + img.Attribute("src").Value() +" \n";

 

PS. 注意head 中页面的编码信息注意转码

CSS选择器 http://www.w3school.com.cn/cssref/css_selectors.asp

WPF C# 获取机器信息

 0x00.添加引用

using System; using System.Collections.Specialized; 
using System.IO; 
using System.Management; 
using System.Net; 
using System.Net.Sockets; 
using System.Windows; 
using System.Windows.Input; 
using System.Windows.Media;

0x01.获取机器名称

string machineName = Environment.MachineName;

0x02.获取MAC地址

string result;
try
{using (ManagementClass mc = new ManagementClass("Win32_NetworkAdapterConfiguration"))
{
using (ManagementObjectCollection moc = mc.GetInstances())
{
string macAddress = "";
foreach (ManagementObject mo in moc)
{
if ((bool)mo["IPEnabled"] == true)
{
macAddress = mo["MacAddress"].ToString();
break;
}
}
result = macAddress;
}
}
}
catch
{
result = "unknown";
}
finally
{
}

0x03.判断32/64位系统

string result;
bool is64OS = Environment.Is64BitOperatingSystem;
if (is64OS == true)
result = "64 bits OS";//64位
else
result = "32 bits OS";//32位

0x04.获取硬盘信息

string HDSN = "";
string HDID = "";

ManagementClass cimobject = new ManagementClass("Win32_DiskDrive");
ManagementObjectCollection moc1 = cimobject.GetInstances();
foreach (ManagementObject mo in moc1)
{
HDSN = (string)mo.Properties["Model"].Value;
}
ManagementClass mc1 = new ManagementClass("Win32_PhysicalMedia");
ManagementObjectCollection moc2 = mc1.GetInstances();

foreach (ManagementObject mo in moc2)
{
HDID = mo.Properties["SerialNumber"].Value.ToString().Trim();
}

0x05.IPV4

string[] GetLocalIpv4()
string Ipv4;
{
IPAddress[] localIPs;
localIPs = Dns.GetHostAddresses(Dns.GetHostName());
StringCollection IpCollection = new StringCollection();
foreach (IPAddress ip in localIPs)
{
if (ip.AddressFamily == AddressFamily.InterNetwork)
IpCollection.Add(ip.ToString());
}
string[] IpArray = new string[IpCollection.Count];
IpCollection.CopyTo(IpArray, 0);
return IpArray;
}

void ShowIP()
{
foreach (string ip in GetLocalIpv4())
{
Ipv4 = ip.ToString();
}
return;
}

ShowIP();

0x06.IPV6

string ip =  "";
IPAddress[] addressList = Dns.GetHostEntry(Dns.GetHostName()).AddressList;
for (int i = 0; i < addressList.Length; i++)
{
    ip = addressList[i].ToString();
}

0x07.获取厂商和机器型号(manufacturer)(model)

string manufacturer;
string model;
ManagementClass mc2 = new ManagementClass("Win32_ComputerSystem");
ManagementObjectCollection moc3 = mc2.GetInstances();

if (moc3.Count != 0)
{
    foreach (ManagementObject mo in mc2.GetInstances())
    {
        manufacturer =mo["Manufacturer"].ToString();///////////制造厂商
    }
}

foreach (ManagementObject m in moc3)
{
    model = m["model"].ToString ();////机器型号
}

 

C# 获取必应每日一图

 0x00.添加引用

using System;
using System.Net.Http;
using System.Threading.Tasks;
using Windows.Networking.BackgroundTransfer;
using Windows.Storage;
using System.Xml;

0x01.模仿请求

public static Task< bool> GetBingPics()
{
    return Task.Run(async () =>
     {
         string temp = "http://cn.bing.com/HPImageArchive.aspx?idx=0&n=1";
         string downtemp = "http://cn.bing.com";
         string baseurlexp = "//images/image/url";
         string copyrightexp = "//images/image/copyright";
         string dateexp = "//images/image/enddate";

         HttpClient request = new HttpClient();
         {
             request.DefaultRequestHeaders.Add("Accept", "application/json, text/javascript, */*; q=0.01");
             request.DefaultRequestHeaders.Add("UserAgent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0");
             request.DefaultRequestHeaders.Add("Accept-Encoding", "zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3");
             request.DefaultRequestHeaders.Add("Accept-Language", "gzip, deflate");
         }//填写请求头

         try
         {
             HttpResponseMessage response = request.GetAsync(temp).Result;
             string xmlstr = response.Content.ReadAsStringAsync().Result;
             XmlDocument doc = new XmlDocument();
             doc.LoadXml(xmlstr);

             XmlElement root = null;
             root = doc.DocumentElement;

             XmlNode xmlNode = root.SelectSingleNode(baseurlexp);
             XmlNode xmlNode1 = root.SelectSingleNode(copyrightexp);
             XmlNode xmlNode2 = root.SelectSingleNode(dateexp);
             string baseurl = xmlNode.InnerText;
             string copyrighturl = xmlNode1.InnerText;
             string date = xmlNode2.InnerText;
             string plus = "." + baseurl.Split('.')[1];

             BackgroundDownloader backgroundDownload = new BackgroundDownloader();

             StorageFolder folder = await KnownFolders.PicturesLibrary.CreateFolderAsync("BingPics", CreationCollisionOption.OpenIfExists);
             StorageFile newFile = await folder.CreateFileAsync(date + plus, CreationCollisionOption.OpenIfExists);

             Uri uri = new Uri(downtemp + baseurl);
             DownloadOperation download = backgroundDownload.CreateDownload(uri, newFile);

             await download.StartAsync();
             return true;
         }
         catch
         {
             return false;
         }
     });
}

 

UWP 判断程序是否为第一次运行

 0x00.添加引用

using System;
using System.Collections.Generic;
using System.Linq;
using Windows.Storage;

0x01.通过判断本地设置文件中是否存在对应项来判断是否为第一次运行

public static bool IsFirstlyRun()
{
    ApplicationDataContainer localSettings = ApplicationData.Current.LocalSettings;
    if (localSettings.Values["BackgroundSource"] == null)
        return true;
    else
        return false;
}