Nicksxs's Blog

spark 的一些粗浅使用经验

工作中学习使用了一下Spark做数据分析,主要是用spark的python接口，首先是pyspark.SparkContext(appName=xxx)，这是初始化一个Spark应用实例或者说会话，不能重复，
返回的实例句柄就可以调用textFile(path)读取文本文件，这里的文本文件可以是HDFS上的文本文件，也可以普通文本文件，但是需要在Spark的所有集群上都存在，否则会
读取失败，parallelize则可以将python生成的集合数据读取后转换成rdd(A Resilient Distributed Dataset (RDD),一种spark下的基本抽象数据集),基于这个RDD就可以做
数据的流式计算，例如map reduce,在Spark中可以非常方便地实现

简单的mapreduce word count示例

1
2
3
4

textFile = sc.parallelize([(1,1), (2,1), (3,1), (4,1), (5,1),(1,1), (2,1), (3,1), (4,1), (5,1)])
data = textFile.reduceByKey(lambda x, y: x + y).collect()
for _ in data:
    print(_)

结果

1
2
3
4
5

(3, 2)
(1, 2)
(4, 2)
(2, 2)
(5, 2)

PHP抽象类和接口

• 抽象类与接口
• 抽象类内可以包含非抽象函数，即可实现函数
• 抽象类内必须包含至少一个抽象方法，抽象类和接口均不能实例化
• 抽象类可以设置访问级别，接口默认都是public
• 类可以实现多个接口但不能继承多个抽象类
• 类必须实现抽象类和接口里的抽象方法，不一定要实现抽象类的非抽象方法
• 接口内不能定义变量，但是可以定义常量

示例代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

<?php
interface int1{
    const INTER1 = 111;
    function inter1();
}
interface int2{
    const INTER1 = 222;
    function inter2();
}
abstract class abst1{
    public function abstr1(){
        echo 1111;
    }
    abstract function abstra1(){
        echo 'ahahahha';
    }
}
abstract class abst2{
    public function abstr2(){
        echo 1111;
    }
    abstract function abstra2();
}
class normal1 extends abst1{
    protected function abstr2(){
        echo 222;
    }
}

result

1
2
3

PHP Fatal error:  Abstract function abst1::abstra1() cannot contain body in new.php on line 17

Fatal error: Abstract function abst1::abstra1() cannot contain body in php on line 17

problem

Given a sorted integer array without duplicates, return the summary of its ranges.

For example, given [0,1,2,4,5,7], return ["0->2","4->5","7"].

题解

每一个区间的起点nums[i]加上j是否等于nums[i+j]
参考

Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

class Solution {
public:
    vector<string> summaryRanges(vector<int>& nums) {
        int i = 0, j = 1, n;
        vector<string> res;
        n = nums.size();
        while(i < n){
            j = 1;
            while(j < n && nums[i+j] - nums[i] == j) j++;
            res.push_back(j <= 1 ? to_string(nums[i]) : to_string(nums[i]) + "->" + to_string(nums[i + j - 1]));
            i += j;
        }
        return res;
    }
};

problem

Given an array of n positive integers and a positive integer s, find the minimal length of a subarray of which the sum ≥ s. If there isn’t one, return 0 instead.

For example, given the array [2,3,1,2,4,3] and s = 7,
the subarray [4,3] has the minimal length under the problem constraint.

题解

参考，滑动窗口，跟之前Data Structure课上的online算法有点像，链接

Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

class Solution {
public:
    int minSubArrayLen(int s, vector<int>& nums) {
        int len = nums.size();
        if(len == 0) return 0;
        int minlen = INT_MAX;
        int sum = 0;
        
        int left = 0;
        int right = -1;
        while(right < len)
        {
            while(sum < s && right < len)
                sum += nums[++right];
            if(sum >= s)
            {
                minlen = minlen < right - left + 1 ? minlen : right - left + 1;
                sum -= nums[left++];
            }
        }
        return minlen > len ? 0 : minlen;
    }
};

problem

A binary watch has 4 LEDs on the top which represent the hours (0-11), and the 6 LEDs on the bottom represent the minutes (0-59).

Each LED represents a zero or one, with the least significant bit on the right.

For example, the above binary watch reads “3:25”.

Given a non-negative integer n which represents the number of LEDs that are currently on, return all possible times the watch could represent.

Example:

1
2

Input: n = 1
Return: ["1:00", "2:00", "4:00", "8:00", "0:01", "0:02", "0:04", "0:08", "0:16", "0:32"]

Note:

• The order of output does not matter.
• The hour must not contain a leading zero, for example “01:00” is not valid, it should be “1:00”.
• The minute must be consist of two digits and may contain a leading zero, for example “10:2” is not valid, it should be “10:02”.

题解

又是参（chao）考（xi）别人的代码，嗯，就是这么不要脸，链接

Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14

class Solution {
public:
    vector<string> readBinaryWatch(int num) {
        vector<string> res;
        for (int h = 0; h < 12; ++h) {
            for (int m = 0; m < 60; ++m) {
                if (bitset<10>((h << 6) + m).count() == num) {
                    res.push_back(to_string(h) + (m < 10 ? ":0" : ":") + to_string(m));
                }
            }
        }
        return res;
    }
};