
上QQ阅读APP看书,第一时间看更新
There's more...
One must be careful not to mix vector facilities provided by the Breeze library with Spark ML vectors. To work with ML library algorithms, you are required to use its native data structures, but you can always convert from ML vectors to Breeze, do all your math operations, and then convert to Spark's desired data structure when using the ML library algorithms (for example, ALS or SVD).
We need the vector and matrix import statements so we can work with the ML library itself, otherwise the Scala vector and matrix will be used by default. This is the source of much confusion when the programs fail to scale on cluster.
The following figure depicts a pictorial view which should help clarify the subject:
