如有翻译问题欢迎评论指出，谢谢。
计算Pytorch模型参数量
-
Fábio Perez asked:
- 怎么计算 Pytorch 模型的参数量？类似 Keras 的
model.count_params()那样的函数。
- 怎么计算 Pytorch 模型的参数量？类似 Keras 的
-
Answers:
-
Fábio Perez – vote: 198
-
Pytorch 没有类似 Keras 计算参数量的函数，但可以通过每个参数组的求和得出参数量：
-
pytorch_total_params = sum(p.numel() for p in model.parameters()) -
如果希望只计算可训练的参数：
-
pytorch_total_params = sum(p.numel() for p in model.parameters() if p.requires_grad) -
参考 Pytorch 论坛回答。
-
注：这个回答是自问自答，如果你们有更好的解决方法，请分享一下。
-
Fábio Perez – vote: 46
-
为了像 Keras 一样计算每层的参数量，Pytorch 的 model.named_paramters() 能返回一个迭代器，包含参数名与参数本身。
-
举例说明：
-
from prettytable import PrettyTable # def count_parameters(model): table = PrettyTable(["Modules", "Parameters"]) total_params = 0 for name, parameter in model.named_parameters(): if not parameter.requires_grad: continue params = parameter.numel() table.add_row([name, params]) total_params+=params print(table) print(f"Total Trainable Params: {total_params}") return total_params # count_parameters(net) -
输出如下：
-
+-------------------+------------+ | Modules | Parameters | +-------------------+------------+ | embeddings.weight | 922866 | | conv1.weight | 1048576 | | conv1.bias | 1024 | | bn1.weight | 1024 | | bn1.bias | 1024 | | conv2.weight | 2097152 | | conv2.bias | 1024 | | bn2.weight | 1024 | | bn2.bias | 1024 | | conv3.weight | 2097152 | | conv3.bias | 1024 | | bn3.weight | 1024 | | bn3.bias | 1024 | | lin1.weight | 50331648 | | lin1.bias | 512 | | lin2.weight | 265728 | | lin2.bias | 519 | +-------------------+------------+ Total Trainable Params: 56773369 -
Thong Nguyen – vote: 12
-
如果希望避免重复计算共享的参数，可以用
torch.Tensor.data_ptr，即： -
sum(dict((p.data_ptr(), p.numel()) for p in model.parameters()).values()) -
下面是更繁琐的实现，添加了一个选项，用于跳过不被训练的参数：
-
def numel(m: torch.nn.Module, only_trainable: bool = False): """ returns the total number of parameters used bym(only counting shared parameters once); ifonly_trainableis True, then only includes parameters withrequires_grad = True""" parameters = list(m.parameters()) if only_trainable: parameters = [p for p in parameters if p.requires_grad] unique = {p.data_ptr(): p for p in parameters}.values() return sum(p.numel() for p in unique)
-
Check the total number of parameters in a PyTorch model
-
Fábio Perez asked:
- How to count the total number of parameters in a PyTorch model? Something similar to
model.count_params()in Keras.
怎么计算 Pytorch 模型的参数量？类似 Keras 的model.count_params()那样的函数。
- How to count the total number of parameters in a PyTorch model? Something similar to
-
Answers:
-
Fábio Perez – vote: 198
-
PyTorch doesn\’t have a function to calculate the total number of parameters as Keras does, but it\’s possible to sum the number of elements for every parameter group:
Pytorch 没有类似 Keras 计算参数量的函数，但可以通过每个参数组的求和得出参数量： -
pytorch_total_params = sum(p.numel() for p in model.parameters()) -
If you want to calculate only the trainable parameters:
如果希望只计算可训练的参数： -
pytorch_total_params = sum(p.numel() for p in model.parameters() if p.requires_grad) -
Answer inspired by this answer on PyTorch Forums.
参考 Pytorch 论坛回答。 -
Note: I\’m answering my own question. If anyone has a better solution, please share with us.
注：这个回答是自问自答，如果你们有更好的解决方法，请分享一下。 -
Fábio Perez – vote: 46
-
To get the parameter count of each layer like Keras, PyTorch has model.named_paramters() that returns an iterator of both the parameter name and the parameter itself.
为了像 Keras 一样计算每层的参数量，Pytorch 的 model.named_paramters() 能返回一个迭代器，包含参数名与参数本身。 -
Here is an example:
举例说明： -
from prettytable import PrettyTable # def count_parameters(model): table = PrettyTable(["Modules", "Parameters"]) total_params = 0 for name, parameter in model.named_parameters(): if not parameter.requires_grad: continue params = parameter.numel() table.add_row([name, params]) total_params+=params print(table) print(f"Total Trainable Params: {total_params}") return total_params # count_parameters(net) -
The output would look something like this:
输出如下： -
+-------------------+------------+ | Modules | Parameters | +-------------------+------------+ | embeddings.weight | 922866 | | conv1.weight | 1048576 | | conv1.bias | 1024 | | bn1.weight | 1024 | | bn1.bias | 1024 | | conv2.weight | 2097152 | | conv2.bias | 1024 | | bn2.weight | 1024 | | bn2.bias | 1024 | | conv3.weight | 2097152 | | conv3.bias | 1024 | | bn3.weight | 1024 | | bn3.bias | 1024 | | lin1.weight | 50331648 | | lin1.bias | 512 | | lin2.weight | 265728 | | lin2.bias | 519 | +-------------------+------------+ Total Trainable Params: 56773369 -
Thong Nguyen – vote: 12
-
If you want to avoid double counting shared parameters, you can use
torch.Tensor.data_ptr. E.g.:
如果希望避免重复计算共享的参数，可以用torch.Tensor.data_ptr，即： -
sum(dict((p.data_ptr(), p.numel()) for p in model.parameters()).values()) -
Here\’s a more verbose implementation that includes an option to filter out non-trainable parameters:
下面是更繁琐的实现，添加了一个选项，用于跳过不被训练的参数： -
def numel(m: torch.nn.Module, only_trainable: bool = False): """ returns the total number of parameters used bym(only counting shared parameters once); ifonly_trainableis True, then only includes parameters withrequires_grad = True""" parameters = list(m.parameters()) if only_trainable: parameters = [p for p in parameters if p.requires_grad] unique = {p.data_ptr(): p for p in parameters}.values() return sum(p.numel() for p in unique)
-
近期评论