深圳幻海软件技术有限公司 欢迎您!

ML .NET 二手车价格预测之再次训练与参数调整(二)

2023-03-22

>再次训练与参数调整在​​UsedCarsPricePredictionMLModel.training.cs​​文件下,有训练设置与训练模型的方法​​BuildPipeline​​方法中是ML.NET自动生成的训练设置,包括选择了哪些参数,预测的字段是什么,以及调用​​LightGbm​​方法,参
>


再次训练与参数调整

在​​UsedCarsPricePredictionMLModel.training.cs​​文件下,有训练设置与训练模型的方法

​BuildPipeline​​方法中是ML .NET自动生成的训练设置,包括选择了哪些参数,预测的字段是什么,

以及调用​​LightGbm​​方法,参数配置为

{
NumberOfLeaves=17,
MinimumExampleCountPerLeaf=25,
NumberOfIterations=6019,
MaximumBinCountPerFeature=24,
LearningRate=1F,
LabelColumnName=@"Price",
FeatureColumnName=@"Features",
Booster=new GradientBooster.Options()
{
SubsampleFraction=0.706948120047722F,
FeatureFraction=0.521537449021549F,
L1Regularization=0.00247814105551342F,
L2Regularization=0.00137211480690565F
}
}

这些都是由ML .NET自动生成好的推荐配置参数,如果本身对机器学习有所研究,可以在此基础上进行修改,以达到优化模型的作用

​​参考资料 LightGbmExtensions.LightGbm 方法​​

完整训练代码如下

public static IEstimator<ITransformer> BuildPipeline(MLContext mlContext)
{
// Data process configuration with pipeline data transformations
var pipeline = mlContext.Transforms.Categorical.OneHotEncoding(new []{new InputOutputColumnPair(@"Fuel_Type", @"Fuel_Type"),new InputOutputColumnPair(@"Transmission", @"Transmission"),new InputOutputColumnPair(@"Owner_Type", @"Owner_Type")})
.Append(mlContext.Transforms.ReplaceMissingValues(new []{new InputOutputColumnPair(@"Year", @"Year"),new InputOutputColumnPair(@"Kilometers_Driven", @"Kilometers_Driven"),new InputOutputColumnPair(@"Seats", @"Seats")}))
.Append(mlContext.Transforms.Text.FeaturizeText(@"Name", @"Name"))
.Append(mlContext.Transforms.Text.FeaturizeText(@"Location", @"Location"))
.Append(mlContext.Transforms.Text.FeaturizeText(@"Engine", @"Engine"))
.Append(mlContext.Transforms.Text.FeaturizeText(@"Power", @"Power"))
.Append(mlContext.Transforms.Concatenate(@"Features", new []{@"Fuel_Type",@"Transmission",@"Owner_Type",@"Year",@"Kilometers_Driven",@"Seats",@"Name",@"Location",@"Engine",@"Power"}))
.Append(mlContext.Regression.Trainers.LightGbm(new LightGbmRegressionTrainer.Options(){NumberOfLeaves=17,MinimumExampleCountPerLeaf=25,NumberOfIterations=6019,MaximumBinCountPerFeature=24,LearningRate=1F,LabelColumnName=@"Price",FeatureColumnName=@"Features",Booster=new GradientBooster.Options(){SubsampleFraction=0.706948120047722F,FeatureFraction=0.521537449021549F,L1Regularization=0.00247814105551342F,L2Regularization=0.00137211480690565F}}));

return pipeline;
}

之后可以调用​​RetrainPipeline​​方法再次训练,得到新的模型

public static ITransformer RetrainPipeline(MLContext context, IDataView trainData)
{
var pipeline = BuildPipeline(context);
var model = pipeline.Fit(trainData);

return model;
}

获取model后保存成文件

//注意,这里使用txt或者tsv格式的文件
string trainCsvPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "TrainData", "train-data.txt");
string testCsvPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "TrainData", "test-data2.txt");
string modelDirectory = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Model");
string modelPath = Path.Combine(modelDirectory, "UsedCarsPricePredictionMLModel.zip");

MLContext mlContext = new MLContext(seed: 0);
IDataView trainingDataView = mlContext.Data.LoadFromTextFile<ModelInput>(trainCsvPath, hasHeader: true);
var model = UsedCarsPricePredictionMLModel.RetrainPipeline(mlContext, trainingDataView);
if (!Directory.Exists(modelDirectory))
Directory.CreateDirectory(modelDirectory);

mlContext.Model.Save(model, trainingDataView.Schema, modelPath);

小问题

问题1:

Property 'Column1' is missing the LoadColumnAttribute attribute

根据提示,需要为​​ModelInput​​模型输入类的每个属性添加​​LoadColumn​​特性,指明所在列

问题2:

Schema mismatch for input column 'Name_CharExtractor': expected Expected known-size vector of Single, got Vector<Single> Arg_ParamName_Name

根据​​ML.NET: Schema mismatch for input column 'AnswerFeaturized_CharExtractor': expected Expected Single or known-size vector of Single, got Vector​​​,不能使用​​.csv​​​文件,改为​​.txt​​​文件或者​​.tsv​​文件

示例代码

​​UsedCarsPricePrediction​​

参考资料

​​10分钟快速入门​​

​​官方示例machinelearning-samples​​

​​教程:将回归与 ML.NET 配合使用以预测价格​​



学习技术最好的文档就是​​【​官方文档​】​​,没有之一。

还有学习资料​​【​Microsoft Learn​】​​、【​CSharp Learn】、​​【​My Note​】​​。

如果,你希望更容易地发现我的新博客,不妨【​关注​】。